Correlating a medical malpractice claim with a portion of a video

ABSTRACT

Disclosed herein are apparatus, system, method, and computer-readable medium aspects for using surgical video analysis for improving compliance with medical guidelines and improving processing of medical bills, medical malpractice claims, and insurance claims. Aspects disclosed herein utilize intracorporeal video footage, image analysis, and notifications to optimize correspondences among medical procedure information, medical guidelines, and medical transaction information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Ser. No. 63/346,987, filed May30, 2022 titled “USAGE AND ANALYSIS OF SURGICAL VIDEOS,” U.S. Ser. No.63/389,130, filed Jul. 14, 2022 titled “SYSTEMS AND METHODS FORCOMPLIANCE IMPROVEMENT IN SURGICAL PROCEDURES,” U.S. Ser. No.63/399,698, filed Aug. 21, 2022 titled “USAGE AND ANALYSIS OF SURGICALVIDEOS FOR COMPLIANCE IMPROVEMENT IN SURGICAL PROCEDURES,” and U.S. Ser.No. 63/411,758, filed Sep. 30, 2022 titled “SYSTEMS AND METHODS FORCOMPLIANCE IMPROVEMENT IN SURGICAL PROCEDURES.” The contents of theforegoing applications are herein incorporated by reference.

TECHNICAL FIELD

Aspects of the present disclosure relate to components, systems, andmethods for analyzing surgical footage. More particularly, aspects ofthe present disclosure relate to components, systems, and methods foranalyzing surgical footage to improve compliance with medical guidelinesand to improve processing of medical bills, medical malpractice claims,and insurance claims.

BACKGROUND

It is important for medical professional, such as surgeons, to followmedical guidelines when performing medical procedures. Accordingly,medical centers or departments are interested in determining whether aselected group of surgeons, such as those who share a particularspecialty, comply with surgical guidelines. Medical centers ordepartments may also wish to check compliance with a customized set ofguidelines chosen from a larger group of guidelines. For example, if ahospital has unusually high post-operative infection rates, the medicalcenters or departments may be particularly interested in guidelines thatimpact infection. Compliance with medical guidelines can also influenceinsurance premiums of medical professionals and medical centers anddepartments.

When medical professionals perform medical procedures, significantamounts of information, such as written, audio, and/or video data, isproduced. If simplified, this information could prove useful whenprocessing various medical transactions. For example, in addition toconsidering insurance premiums, medical malpractice and insurance claimsare a common transactional component of the medical field. With thesignificant amount of information resulting from a medical procedure,processing these transactions can be tedious and error-prone.

SUMMARY

In an aspect, a non-transitory computer readable medium can containinstructions that, when executed by at least one processor, cause the atleast one processor to execute operations to perform intracorporealvideo analysis using a medical malpractice claim. In the aspect, amedical malpractice claim related to a particular surgical procedure isreceived. A linguistic analysis can then be performed on the medicalmalpractice claim. The linguistic analysis can be used to identify asurgical event giving rise to the medical malpractice claim. Then, anintracorporeal video stream depicting the particular surgical procedureis accessed. Based on the earlier identified surgical event, theintracorporeal video stream can be analyzed to identify a series offrames from the intracorporeal video stream that depict the surgicalevent. Based on the identified series of frames, an action correspondingto the medical malpractice claim can be initiated.

System, device, and computer program product aspects are also disclosed.

Further features and advantages, as well as the structure and operationof various aspects, are described in detail below with reference to theaccompanying drawings. It is noted that the specific aspects describedherein are not intended to be limiting. Such aspects are presentedherein for illustrative purposes only. Additional aspects will beapparent to persons skilled in the relevant art(s) based on theteachings contained herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and form a partof the specification, illustrate aspects of the present disclosure and,together with the description, further serve to explain the principlesof the disclosure and to enable a person skilled in the pertinent art tomake and use the disclosure.

FIG. 1 is a perspective view of an example operating room, according tosome aspects of the present disclosure.

FIG. 2 is a perspective view of cameras, according to some aspects ofthe present disclosure.

FIG. 3 is a perspective view of an example of a surgical instrument,according to some aspects of the present disclosure.

FIG. 4 is an illustration of a video of a surgical procedure with anoverlaid timeline, according to some aspects of the present disclosure.

FIG. 5 is a flowchart of an example process for determining groupcompliance using surgical guidelines, according to some aspects of thepresent disclosure.

FIG. 6 is an illustration of a surgical guideline, according to someaspects of the present disclosure.

FIG. 7 is an illustration of a schedule of surgical procedures,according to some aspects of the present disclosure.

FIGS. 8A-8C are illustrations of intracorporeal video streams, accordingto some aspects of the present disclosure.

FIG. 9 is a flowchart of an example process for determining compliancewith selected surgical guidelines, according to some aspects of thepresent disclosure.

FIG. 10 is an illustration of groups of surgical guidelines, accordingto some aspects of the present disclosure.

FIG. 11 is a flowchart of an example process of surgical video analysisfor insurance premium adjustment, according to some aspects of thepresent disclosure.

FIG. 12 is a flowchart of an example process for correlating a medicalmalpractice claim with a portion of a video, according to some aspectsof the present disclosure.

FIG. 13 is a flowchart of an example linguistic analysis performed on amedical malpractice claim to identify a surgical event, according tosome aspects of the present disclosure.

FIG. 14 is an illustration of intracorporeal video streams, according tosome aspects of the present disclosure.

FIG. 15 is a flowchart of an example process for correlating a medicalclaim code with a portion of a video, according to some aspects of thepresent disclosure.

FIG. 16 is an illustration of an example process for correlating amedical claim code with a portion of a video, according to some aspectsof the present disclosure.

FIG. 17 is a flowchart of an example process for analyzing surgicalvideo to support insurance reimbursement, according to some aspects ofthe present disclosure.

FIG. 18 is an illustration of an example process for analyzing surgicalvideo to support insurance reimbursement, according to some aspects ofthe present disclosure.

FIG. 19 is a flowchart of an example process for analyzing surgicalvideos to identify a billing coding mismatch, according to some aspectsof the present disclosure.

FIG. 20 is a block diagram of an example computer system useful forimplementing various aspects.

In the drawings, like reference numbers generally indicate identical orsimilar elements. Additionally, generally, the left-most digit(s) of areference number identifies the drawing in which the reference numberfirst appears.

Aspects of the present disclosure will be described with reference tothe accompanying drawings.

DETAILED DESCRIPTION

FIG. 1 shows an example operating room 101, consistent with disclosedaspects. A patient 143 is illustrated on an operating table 141. Room101 may include audio sensors, video/image sensors, chemical sensors,and other sensors, as well as various light sources (e.g., light source119) for facilitating capture of video and audio data, as well as datafrom other sensors, during the surgical procedure. For example, room 101may include one or more microphones (e.g., audio sensor 111), severalcameras (e.g., overhead cameras 115, 121, and 123, and a tablesidecamera 125) for capturing video/image data during surgery. While some ofthe cameras (e.g., cameras 115, 123, and 125) may capture video/imagedata of operating table 141 (e.g., the cameras may capture thevideo/image data at a location 127 of a body of patient 143 on which asurgical procedure is performed), camera 121 may capture video/imagedata of other parts of operating room 101. For instance, camera 121 maycapture video/image data of a surgeon 131 performing the surgery. Insome cases, cameras may capture video/image data of surgical teampersonnel, such as an anesthesiologist, nurses, surgical tech and thelike located in operating room 101. Additionally, operating room camerasmay capture video/image data of medical equipment located in the room.During surgery, some of the cameras (e.g., cameras 115, 121, 123, and125) may capture intracorporeal video footage.

In various aspects, one or more of cameras 115, 121, 123, and 125 may bemovable. For example, as shown in FIG. 1 , camera 115 may be rotated asindicated by arrows 135A showing a pitch direction, and arrows 135Bshowing a yaw direction for camera 115. In various aspects, pitch andyaw angles of cameras (e.g., camera 115) may be electronicallycontrolled such that camera 115 points at a region-of-interest (ROI), ofwhich video/image data needs to be captured. For example, camera 115 maybe configured to track a surgical instrument (also referred to as asurgical tool) within location 127, an anatomical structure, a hand ofsurgeon 131, an incision, a movement of anatomical structure, and thelike. In various aspects, camera 115 may be equipped with a laser 137(e.g., an infrared laser) for precision tracking. In some cases, camera115 may be tracked automatically via a computer-based camera controlapplication that uses an image recognition algorithm for positioning thecamera to capture video/image data of a ROI. For example, the cameracontrol application may identify an anatomical structure, identify asurgical tool, hand of a surgeon, bleeding, motion, and the like at aparticular location within the anatomical structure, and track thatlocation with camera 115 by rotating camera 115 by appropriate yaw andpitch angles. In some aspects, the camera control application maycontrol positions (i.e., yaw and pitch angles) of various cameras 115,121, 123, and 125 to capture video/image date from different ROIs duringa surgical procedure. Additionally or alternatively, a human operatormay control the position of various cameras 115, 121, 123, and 125,and/or the human operator may supervise the camera control applicationin controlling the position of the cameras.

Cameras 115, 121, 123, and 125 may further include zoom lenses forfocusing in on and magnifying one or more ROIs. In an example aspect,camera 115 may include a zoom lens 138 for zooming closely to a ROI(e.g., a surgical tool in the proximity of an anatomical structure).Camera 121 may include a zoom lens 139 for capturing video/image datafrom a larger area around the ROI. For example, camera 121 may capturevideo/image data for the entire location 127. In some aspects,video/image data obtained from camera 121 may be analyzed to identify aROI during the surgical procedure, and the camera control applicationmay be configured to cause camera 115 to zoom towards the ROI identifiedby camera 121.

In various aspects, the camera control application may be configured tocoordinate the position, focus, and magnification of various camerasduring a surgical procedure. For example, the camera control applicationmay direct camera 115 to track an anatomical structure and may directcamera 121 and 125 to track a surgical instrument. Cameras 121 and 125may track the same ROI (e.g., a surgical instrument) from different viewangles. For example, video/image data obtained from different viewangles may be used to determine the position of the surgical instrumentrelative to a surface of the anatomical structure, to determine acondition of an anatomical structure, to determine pressure applied toan anatomical structure, or to determine any other information wheremultiple viewing angles may be beneficial. By way of another example,bleeding may be detected by one camera, and one or more other camerasmay be used to identify the source of the bleeding.

In various aspects, control of position, orientation, settings, and/orzoom of cameras 115, 121, 123, and 125 may be rule-based and follow analgorithm developed for a given surgical procedure. For example, thecamera control application may be configured to direct camera 115 totrack a surgical instrument, to direct camera 121 to location 127, todirect camera 123 to track the motion of the surgeon's hands, and todirect camera 125 to an anatomical structure. The algorithm may includeany suitable logical statements determining position, orientation,settings and/or zoom for cameras 115, 121, 123, and 125 depending onvarious events during the surgical procedure. For example, the algorithmmay direct at least one camera to a region of an anatomical structurethat develops bleeding during the procedure. Some non-limiting examplesof settings of cameras 115, 121, 123, and 125 that may be controlled(for example by the camera control application) may include image pixelresolution, frame rate, image and/or color correction and/or enhancementalgorithms, zoom, position, orientation, aspect ratio, shutter speed,aperture, focus, and so forth.

In various cases, when a camera (e.g., camera 115) tracks a moving ordeforming object (e.g., when camera 115 tracks a moving surgicalinstrument, or a moving/pulsating anatomical structure), a cameracontrol application may determine a maximum allowable zoom for camera115, such that the moving or deforming object does not escape a field ofview of the camera. In an example aspect, the camera control applicationmay initially select the first zoom for camera 115, evaluate whether themoving or deforming object escapes the field of view of the camera, andadjust the zoom of the camera as necessary to prevent the moving ordeforming object from escaping the field of view of the camera. Invarious aspects, the camera zoom may be readjusted based on a directionand a speed of the moving or deforming object.

In various aspects, one or more image sensors may include moving cameras115, 121, 123, and 125. Cameras 115, 121, 123, and 125 may be used fordetermining sizes of anatomical structures and determining distancesbetween different ROIs, for example using triangulation. For example,FIG. 2 shows exemplary cameras 115 and 121 supported by movable elementssuch that the distance between the two cameras is D₁, as shown in FIG. 2. Both cameras point at ROI 223. By knowing the positions of cameras 115and 121 and the direction of an object relative to the cameras (e.g., byknowing angles A₁ and A₂, as shown in FIG. 2 , for example based oncorrespondences between pixels depicting the same object or the samereal-world point in the images captured by 115 and 121), distances D₂and D₃ may be calculated using, for example, the law of sines and theknown distance between the two cameras D₁. In an example aspect, whencamera 115 rotates by a small angle A₃ (measured in radians), to pointat ROI 225, the distance between ROI 223 and ROI 225 may be approximated(for small angles A₃) by A₃D₂. More accuracy may be obtained usinganother triangulation process.

In some aspects, the operating room may include sensors embedded invarious components depicted or not depicted in FIG. 1 . Examples of suchsensors may include: audio sensors; image sensors; motion sensors;positioning sensors; chemical sensors; temperature sensors; barometers;pressure sensors; proximity sensors; electrical impedance sensors;electrical voltage sensors; electrical current sensors; or any otherdetector capable of providing feedback on the environment or a surgicalprocedure, including, for example, any kind of medical or physiologicalsensor configured to monitor patient 143.

In various aspects, temperature sensors may include infrared cameras(e.g., an infrared camera 117) for thermal imaging. Infrared camera 117may allow measurements of the surface temperature of an anatomicstructure at different points of the structure. Similar to visiblecameras 115, 121, 123, and 125, infrared camera 117 may be rotated usingyaw or pitch angles, and may be used to capture intracorporeal videofootage. Additionally or alternatively, camera 117 may include an imagesensor configured to capture image from any light spectrum, includeinfrared image sensor, hyper-spectral image sensors, and so forth.

FIG. 1 includes a display screen 113 that may show views from differentcameras 115, 121, 123, and 125, as well as other information. Forexample, display screen 113 may show a zoomed-in image of a tip of asurgical instrument and a surrounding tissue of an anatomical structurein proximity to the surgical instrument.

FIG. 3 shows an example aspect of a surgical instrument 301 that mayinclude multiple sensors and light-emitting sources. Consistent with thepresent aspects, a surgical instrument may refer to a medical device, amedical instrument, an electrical or mechanical tool, a surgical tool, adiagnostic tool, and/or any other instrumentality that may be usedduring a surgery. As shown, instrument 301 may include cameras 311A and311B, light sources 313A and 313B as well as tips 323A and 323B forcontacting tissue 331. During surgery, cameras 311A and 311B may captureintracorporeal video footage. Cameras 311A and 311B may be connected viadata connection 319A and 319B to a data transmitting device 321. In anexample aspect, device 321 may transmit data to a data-receiving deviceusing a wireless communication or using a wired communication. In anexample aspect, device 321 may use WiFi, Bluetooth, NFC communication,inductive communication, or any other suitable wireless communicationfor transmitting data to a data-receiving device. The data-receivingdevice may include any form of receiver capable of receiving datatransmissions. Additionally or alternatively, device 321 may use opticalsignals to transmit data to the data-receiving device (e.g., device 321may use optical signals transmitted through the air or via opticalfiber). In some aspects, device 301 may include local memory for storingat least some of the data received from cameras 311A and 311B.Additionally, device 301 may include a processor for compressingvideo/image data before transmitting the data to the data-receivingdevice.

In various aspects, for example when device 301 is wireless, it mayinclude an internal power source (e.g., a battery, a rechargeablebattery, and the like) and/or a port for recharging the battery, anindicator for indicating the amount of power remaining for the powersource, and one or more input controls (e.g., buttons) for controllingthe operation of device 301. In some aspects, control of device 301 maybe accomplished using an external computing device (e.g., a smartphone,tablet, smart glasses) communicating with device 301 via any suitableconnection (e.g., WiFi, Bluetooth, and the like). In an example aspect,input controls for device 301 may be used to control various parametersof sensors or light sources. Additionally, instrument 301 may beconfigured to measure data related to various properties of tissue 331via tips 323A and 323B and transmit the measured data to device 321. Forexample, tips 323A and 323B may be used to measure the electricalresistance and/or impedance of tissue 331, the temperature of tissue331, mechanical properties of tissue 331 and the like. To determineelastic properties of tissue 331, for example, tips 323A and 323B may befirst separated by an angle 317 and applied to tissue 331. The tips maybe configured to move such as to reduce angle 317, and the motion oftips may result in pressure on tissue 331. Such pressure may be measured(e.g., via a piezoelectric element 327 that may be located between afirst branch 312A and a second branch 312B of instrument 301), and basedon the change in angle 317 (i.e., strain) and the measured pressure(i.e., stress), the elastic properties of tissue 331 may be measured.Furthermore, based on angle 317 distance between tips 323A and 323B maybe measured, and this distance may be transmitted to device 321. Suchdistance measurements may be used as a length scale for variousvideo/image data that may be captured by various cameras 115, 121, 123,and 125, as shown in FIG. 1 .

Instrument 301 is only one example of possible surgical instrument, andother surgical instruments such as scalpels, graspers (e.g., forceps),clamps and occluders, needles, retractors, cutters, dilators, suctiontips, and tubes, sealing devices, irrigation and injection needles,scopes and probes, and the like, may include any suitable sensors andlight-emitting sources. In various cases, the type of sensors andlight-emitting sources may depend on a type of surgical instrument usedfor a surgical procedure. In various cases, these other surgicalinstruments may include a device similar to device 301, as shown in FIG.3 , for collecting and transmitting data to any suitable data-receivingdevice.

Consistent with disclosed aspects, a method may involve accessing atleast one video of a surgical procedure. The video may include any formof recorded visual media including recorded images and/or sound. Thevideo may be stored as a video file such as an Audio Video Interleave(AVI) file, a Flash Video Format (FLV) file, QuickTime File Format(MOV), MPEG (MPG, MP4, M4P, etc.), a Windows Media Video (WMV) file, aMaterial Exchange Format (MXF) file, or any other suitable video fileformats, for example as described above.

A surgical procedure may include any medical procedure involving manualor operative procedures on a patient's body. Surgical procedures mayinclude cutting, abrading, suturing, or other techniques that involvephysically changing body tissues and organs. A video of a surgicalprocedure may include any series of still images that were capturedduring the surgical procedure. In some aspects, at least a portion ofthe surgical procedure may be depicted in one or more of the stillimages included in the video. For example, the video of the surgicalprocedure may be recorded by an image capture device, such as a camera,in an operating room or in a cavity of a patient. Accessing the video ofthe surgical procedure may include retrieving the video from a storagedevice (such as one or more memory units, a video server, a cloudstorage platform, or any other storage platform), receiving the videofrom another device through a communication device, capturing the videousing image sensors, or any other means for electronically accessingdata or files.

Some aspects of the present disclosure may involve causing the at leastone video to be output for display. Outputting the at least one videomay include any process by which the video is produced, delivered, orsupplied using a computer or at least one processor. As used herein,“display” may refer to any manner in which a video may be presented to auser for playback. In some aspects, outputting the video may includepresenting the video using a display device, such as a screen (e.g., anOLED, QLED LCD, plasma, CRT, DLPT, electronic paper, or similar displaytechnology), a light projector (e.g., a movie projector, a slideprojector), a 3D display, screen of a mobile device, electronic glassesor any other form of visual and/or audio presentation. In other aspects,outputting the video for display may include storing the video in alocation that is accessible by one or more other computing devices. Suchstorage locations may include a local storage (such as a hard drive offlash memory), a network location (such as a server or database), acloud computing platform, or any other accessible storage location. Thevideo may be accessed from a separate computing device for display onthe separate computing device. In some aspects, outputting the video mayinclude transmitting the video to an external computing device. Forexample, outputting the video for display may include transmitting thevideo through a network to a user device for playback on the userdevice.

FIG. 4 illustrates presenting video in a video playback region 410,which may sequentially display one or more frames of the video. Atimeline 420 may be overlaid on the video.

In some embodiments, machine learning algorithms (also referred to asmachine learning models in the present disclosure) may be trained usingtraining examples, for example in the cases described below. Somenon-limiting examples of such machine learning algorithms may includeclassification algorithms, data regressions algorithms, imagesegmentation algorithms, visual detection algorithms (such as objectdetectors, face detectors, person detectors, motion detectors, edgedetectors, etc.), visual recognition algorithms (such as facerecognition, person recognition, object recognition, etc.), speechrecognition algorithms, mathematical embedding algorithms, naturallanguage processing algorithms, support vector machines, random forests,nearest neighbors algorithms, deep learning algorithms, artificialneural network algorithms, convolutional neural network algorithms,recurrent neural network algorithms, linear machine learning models,non-linear machine learning models, ensemble algorithms, and so forth.For example, a trained machine learning algorithm may comprise aninference model, such as a predictive model, a classification model, adata regression model, a clustering model, a segmentation model, anartificial neural network (such as a deep neural network, aconvolutional neural network, a recurrent neural network, etc.), arandom forest, a support vector machine, and so forth. In some examples,the training examples may include example inputs together with thedesired outputs corresponding to the example inputs. Further, in someexamples, training machine learning algorithms using the trainingexamples may generate a trained machine learning algorithm, and thetrained machine learning algorithm may be used to estimate outputs forinputs not included in the training examples. In some examples,engineers, scientists, processes and machines that train machinelearning algorithms may further use validation examples and/or testexamples. For example, validation examples and/or test examples mayinclude example inputs together with the desired outputs correspondingto the example inputs, a trained machine learning algorithm and/or anintermediately trained machine learning algorithm may be used toestimate outputs for the example inputs of the validation examplesand/or test examples, the estimated outputs may be compared to thecorresponding desired outputs, and the trained machine learningalgorithm and/or the intermediately trained machine learning algorithmmay be evaluated based on a result of the comparison. In some examples,a machine learning algorithm may have parameters and hyper parameters,where the hyper parameters may be set manually by a person orautomatically by an process external to the machine learning algorithm(such as a hyper parameter search algorithm), and the parameters of themachine learning algorithm may be set by the machine learning algorithmbased on the training examples. In some implementations, thehyper-parameters may be set based on the training examples and thevalidation examples, and the parameters may be set based on the trainingexamples and the selected hyper-parameters. For example, given thehyper-parameters, the parameters may be conditionally independent of thevalidation examples.

In some embodiments, trained machine learning algorithms (also referredto as machine learning models and trained machine learning models in thepresent disclosure) may be used to analyze inputs and generate outputs,for example in the cases described below. In some examples, a trainedmachine learning algorithm may be used as an inference model that whenprovided with an input generates an inferred output. For example, atrained machine learning algorithm may include a classificationalgorithm, the input may include a sample, and the inferred output mayinclude a classification of the sample (such as an inferred label, aninferred tag, and so forth). In another example, a trained machinelearning algorithm may include a regression model, the input may includea sample, and the inferred output may include an inferred valuecorresponding to the sample. In yet another example, a trained machinelearning algorithm may include a clustering model, the input may includea sample, and the inferred output may include an assignment of thesample to at least one cluster. In an additional example, a trainedmachine learning algorithm may include a classification algorithm, theinput may include an image, and the inferred output may include aclassification of an item depicted in the image. In yet another example,a trained machine learning algorithm may include a regression model, theinput may include an image, and the inferred output may include aninferred value corresponding to an item depicted in the image (such asan estimated property of the item, such as size, volume, age of a persondepicted in the image, cost of a product depicted in the image, and soforth). In an additional example, a trained machine learning algorithmmay include an image segmentation model, the input may include an image,and the inferred output may include a segmentation of the image. In yetanother example, a trained machine learning algorithm may include anobject detector, the input may include an image, and the inferred outputmay include one or more detected objects in the image and/or one or morelocations of objects within the image. In some examples, the trainedmachine learning algorithm may include one or more formulas and/or oneor more functions and/or one or more rules and/or one or moreprocedures, the input may be used as input to the formulas and/orfunctions and/or rules and/or procedures, and the inferred output may bebased on the outputs of the formulas and/or functions and/or rulesand/or procedures (for example, selecting one of the outputs of theformulas and/or functions and/or rules and/or procedures, using astatistical measure of the outputs of the formulas and/or functionsand/or rules and/or procedures, and so forth).

In some embodiments, artificial neural networks may be configured toanalyze inputs and generate corresponding outputs, for example in thecases described below. Some non-limiting examples of such artificialneural networks may comprise shallow artificial neural networks, deepartificial neural networks, feedback artificial neural networks, feedforward artificial neural networks, autoencoder artificial neuralnetworks, probabilistic artificial neural networks, time delayartificial neural networks, convolutional artificial neural networks,recurrent artificial neural networks, long short term memory artificialneural networks, and so forth. In some examples, an artificial neuralnetwork may be configured manually. For example, a structure of theartificial neural network may be selected manually, a type of anartificial neuron of the artificial neural network may be selectedmanually, a parameter of the artificial neural network (such as aparameter of an artificial neuron of the artificial neural network) maybe selected manually, and so forth. In some examples, an artificialneural network may be configured using a machine learning algorithm. Forexample, a user may select hyper-parameters for the an artificial neuralnetwork and/or the machine learning algorithm, and the machine learningalgorithm may use the hyper-parameters and training examples todetermine the parameters of the artificial neural network, for exampleusing back propagation, using gradient descent, using stochasticgradient descent, using mini-batch gradient descent, and so forth. Insome examples, an artificial neural network may be created from two ormore other artificial neural networks by combining the two or more otherartificial neural networks into a single artificial neural network.

In some embodiments, generative models may be configured to generate newcontent, such as textual content, visual content, auditory content,graphical content, and so forth. In some examples, generative models maygenerate new content without input. In other examples, generative modelsmay generate new content based on an input. In one example, the newcontent may be fully determined from the input, where every usage of thegenerative model with the same input will produce the same new content.In another example, the new content may be associated with the input butnot fully determined from the input, where every usage of the generativemodel with the same input may product a different new content that isassociated with the input. In some examples, a generative model may be aresult of training a machine learning generative algorithm with trainingexamples. An example of such training example may include a sampleinput, together with a sample content associated with the sample input.Some non-limiting examples of such generative models may include DeepGenerative Model (DGM), Generative Adversarial Network model (GAN),auto-regressive model, Variational AutoEncoder (VAE), transformers basedgenerative model, artificial neural networks based generative model,hard-coded generative model, and so forth.

Some non-limiting examples of audio data may include audio recordings,audio stream, audio data that includes speech, audio data that includesmusic, audio data that includes ambient noise, digital audio data,analog audio data, digital audio signals, analog audio signals, monoaudio data, stereo audio data, surround audio data, audio data capturedusing at least one audio sensor, audio data generated artificially, andso forth. In one example, audio data may be generated artificially fromtextual content, for example using text-to-speech algorithms. In anotherexample, audio data may be generated using a generative machine learningmodel. In some embodiments, analyzing audio data (for example, by themethods, steps and modules described herein) may comprise analyzing theaudio data to obtain a preprocessed audio data, and subsequentlyanalyzing the audio data and/or the preprocessed audio data to obtainthe desired outcome. One of ordinary skill in the art will recognizethat the followings are examples, and that the audio data may bepreprocessed using other kinds of preprocessing methods. In someexamples, the audio data may be preprocessed by transforming the audiodata using a transformation function to obtain a transformed audio data,and the preprocessed audio data may comprise the transformed audio data.For example, the transformation function may comprise a multiplicationof a vectored time series representation of the audio data with atransformation matrix. For example, the transformation function maycomprise convolutions, audio filters (such as low-pass filters,high-pass filters, band-pass filters, all-pass filters, etc.), linearfunctions, nonlinear functions, and so forth. In some examples, theaudio data may be preprocessed by smoothing the audio data, for exampleusing Gaussian convolution, using a median filter, and so forth. In someexamples, the audio data may be preprocessed to obtain a differentrepresentation of the audio data. For example, the preprocessed audiodata may comprise: a representation of at least part of the audio datain a frequency domain; a Discrete Fourier Transform of at least part ofthe audio data; a Discrete Wavelet Transform of at least part of theaudio data; a time/frequency representation of at least part of theaudio data; a spectrogram of at least part of the audio data; a logspectrogram of at least part of the audio data; a Mel-Frequency Spectrumof at least part of the audio data; a sonogram of at least part of theaudio data; a periodogram of at least part of the audio data; arepresentation of at least part of the audio data in a lower dimension;a lossy representation of at least part of the audio data; a losslessrepresentation of at least part of the audio data; a time order seriesof any of the above; any combination of the above; and so forth. In someexamples, the audio data may be preprocessed to extract audio featuresfrom the audio data. Some non-limiting examples of such audio featuresmay include: auto-correlation; number of zero crossings of the audiosignal; number of zero crossings of the audio signal centroid; MP3 basedfeatures; rhythm patterns; rhythm histograms; spectral features, such asspectral centroid, spectral spread, spectral skewness, spectralkurtosis, spectral slope, spectral decrease, spectral roll-off, spectralvariation, etc.; harmonic features, such as fundamental frequency,noisiness, inharmonicity, harmonic spectral deviation, harmonic spectralvariation, tristimulus, etc.; statistical spectrum descriptors; waveletfeatures; higher level features; perceptual features, such as totalloudness, specific loudness, relative specific loudness, sharpness,spread, etc.; energy features, such as total energy, harmonic partenergy, noise part energy, etc.; temporal features; and so forth. Insome examples, analyzing the audio data may include calculating at leastone convolution of at least a portion of the audio data, and using thecalculated at least one convolution to calculate at least one resultingvalue and/or to make determinations, identifications, recognitions,classifications, and so forth.

In some embodiments, analyzing audio data (for example, in the casesdescribed below) may comprise analyzing the audio data and/or thepreprocessed audio data using one or more rules, functions, procedures,artificial neural networks, speech recognition algorithms, speakerrecognition algorithms, speaker diarization algorithms, audiosegmentation algorithms, noise cancelling algorithms, source separationalgorithms, inference models, and so forth. Some non-limiting examplesof such inference models may include: an inference model preprogrammedmanually; a classification model; a data regression model; a result oftraining algorithms, such as machine learning algorithms and/or deeplearning algorithms, on training examples, where the training examplesmay include examples of data instances, and in some cases, a datainstance may be labeled with a corresponding desired label and/orresult; and so forth.

Some non-limiting examples of image data may include images, grayscaleimages, color images, 2D images, 3D images, videos, 2D videos, 3Dvideos, frames, footages, data derived from other image data, and soforth. In some embodiments, analyzing image data (for example in thecases described below) may comprise analyzing the image data to obtain apreprocessed image data, and subsequently analyzing the image dataand/or the preprocessed image data to obtain the desired outcome. One ofordinary skill in the art will recognize that the followings areexamples, and that the image data may be preprocessed using other kindsof preprocessing methods. In some examples, the image data may bepreprocessed by transforming the image data using a transformationfunction to obtain a transformed image data, and the preprocessed imagedata may comprise the transformed image data. For example, thetransformed image data may comprise one or more convolutions of theimage data. For example, the transformation function may comprise one ormore image filters, such as low-pass filters, high-pass filters,band-pass filters, all-pass filters, and so forth. In some examples, thetransformation function may comprise a nonlinear function. In someexamples, the image data may be preprocessed by smoothing at least partsof the image data, for example using Gaussian convolution, using amedian filter, and so forth. In some examples, the image data may bepreprocessed to obtain a different representation of the image data. Forexample, the preprocessed image data may comprise: a representation ofat least part of the image data in a frequency domain; a DiscreteFourier Transform of at least part of the image data; a Discrete WaveletTransform of at least part of the image data; a time/frequencyrepresentation of at least part of the image data; a representation ofat least part of the image data in a lower dimension; a lossyrepresentation of at least part of the image data; a losslessrepresentation of at least part of the image data; a time ordered seriesof any of the above; any combination of the above; and so forth. In someexamples, the image data may be preprocessed to extract edges, and thepreprocessed image data may comprise information based on and/or relatedto the extracted edges. In some examples, the image data may bepreprocessed to extract image features from the image data. Somenon-limiting examples of such image features may comprise informationbased on and/or related to: edges; corners; blobs; ridges; ScaleInvariant Feature Transform (SIFT) features; temporal features; and soforth. In some examples, analyzing the image data may includecalculating at least one convolution of at least a portion of the imagedata, and using the calculated at least one convolution to calculate atleast one resulting value and/or to make determinations,identifications, recognitions, classifications, and so forth.

In some embodiments, analyzing image data (for example in the casesdescribed below) may comprise analyzing the image data and/or thepreprocessed image data using one or more rules, functions, procedures,artificial neural networks, object detection algorithms, face detectionalgorithms, visual event detection algorithms, action detectionalgorithms, motion detection algorithms, background subtractionalgorithms, inference models, and so forth. Some non-limiting examplesof such inference models may include: an inference model preprogrammedmanually; a classification model; a regression model; a result oftraining algorithms, such as machine learning algorithms and/or deeplearning algorithms, on training examples, where the training examplesmay include examples of data instances, and in some cases, a datainstance may be labeled with a corresponding desired label and/orresult; and so forth. In some embodiments, analyzing image data (forexample in the cases described below) may comprise analyzing pixels,voxels, point cloud, range data, etc. included in the image data.

A convolution may include a convolution of any dimension. Aone-dimensional convolution is a function that transforms an originalsequence of numbers to a transformed sequence of numbers. Theone-dimensional convolution may be defined by a sequence of scalars.Each particular value in the transformed sequence of numbers may bedetermined by calculating a linear combination of values in asubsequence of the original sequence of numbers corresponding to theparticular value. A result value of a calculated convolution may includeany value in the transformed sequence of numbers. Likewise, ann-dimensional convolution is a function that transforms an originaln-dimensional array to a transformed array. The n-dimensionalconvolution may be defined by an n-dimensional array of scalars (knownas the kernel of the n-dimensional convolution). Each particular valuein the transformed array may be determined by calculating a linearcombination of values in an n-dimensional region of the original arraycorresponding to the particular value. A result value of a calculatedconvolution may include any value in the transformed array. In someexamples, an image may comprise one or more components (such as colorcomponents, depth component, etc.), and each component may include a twodimensional array of pixel values. In one example, calculating aconvolution of an image may include calculating a two dimensionalconvolution on one or more components of the image. In another example,calculating a convolution of an image may include stacking arrays fromdifferent components to create a three dimensional array, andcalculating a three dimensional convolution on the resulting threedimensional array. In some examples, a video may comprise one or morecomponents (such as color components, depth component, etc.), and eachcomponent may include a three dimensional array of pixel values (withtwo spatial axes and one temporal axis). In one example, calculating aconvolution of a video may include calculating a three dimensionalconvolution on one or more components of the video. In another example,calculating a convolution of a video may include stacking arrays fromdifferent components to create a four dimensional array, and calculatinga four dimensional convolution on the resulting four dimensional array.In some examples, audio data may comprise one or more channels, and eachchannel may include a stream or a one-dimensional array of values. Inone example, calculating a convolution of audio data may includecalculating a one dimensional convolution on one or more channels of theaudio data. In another example, calculating a convolution of audio datamay include stacking arrays from different channels to create a twodimensional array, and calculating a two dimensional convolution on theresulting two dimensional array.

FIG. 5 is a flowchart of an example process 500 for determining groupcompliance using surgical guidelines, according to some aspects of thepresent disclosure. It is to be appreciated that not all steps can beneeded to perform the disclosure provided herein. Further, some of thesteps can be performed simultaneously, or in a different order thanshown in FIG. 5 , as will be understood by a person of ordinary skill inthe art.

Process 500 can be implemented by devices and systems described in FIGS.1-4 and using operations caused by computer system 2000. Process 500 canalso be understood with reference to FIGS. 6-8 . However, process 500 isnot limited to these example aspects.

Medical centers or departments may be interested in determining whethera selected group of surgeons, such as those who share a particularspecialty, comply with surgical guidelines. Intracorporeal video footagemay be analyzed to determine a group compliance level. For example, insome aspects, a non-transitory computer readable medium can containinstructions that, when executed by at least one processor, cause the atleast one processor to execute operations to perform intracorporealvideo analysis for monitoring compliance with surgical guidelines. Inone example, a compliance level (such as the group compliance level, anaggregated compliance level, an individual compliance level of aparticular surgeon, a compliance level for a particular surgicalprocedure, etc.) may be a numerical value. In another example, thecompliance level may be a discrete value. In another example, thecompliance level may be a continuous numerical value. In yet anotherexample, the compliance level may be on a categorical scale (such as‘High’, ‘Medium’, low′, ‘None’, and so forth).

In 502, a data structure identifying a particular surgical guideline isaccessed. For example, at least part of the data structure may be readfrom memory, may be received from an external computing device (e.g.,using a digital communication device), and so forth. The particularsurgical guideline may specify a set of actions that are to take placeduring a particular type of surgical procedure. The data structure mayidentify a plurality of surgical guidelines, with each surgicalguideline specifying a set of actions to take place during a type ofsurgical procedure. For example, such set of actions may include asingle action, at least two actions, between three and five actions,between six and ten actions, more than ten actions, and so forth. In oneexample, such surgical guideline may further specify one or more actionsto be avoided during the respective type of surgical procedure. In oneexample, the surgical guideline may further indicate, for each action ofthe set of actions, a level indicative of the importance and/orcriticality of the respective action.

FIG. 6 illustrates a particular surgical guideline, according to someaspects of the present disclosure. As illustrated, the particularsurgical guideline specifies a set of actions, Action 1, Action 2, andAction 3, related to an example surgical procedure, laparoscopiccholecystectomy.

Returning to FIG. 5 , in 504, a group of surgeons who performed aplurality of surgical procedures is identified. For example, the groupof surgeons can be identified from a memory, from an external computingdevice (e.g., using a digital communication device), received from auser (e.g., via a user interface), by accessing a database, by accessinga data-structure associating surgical procedures with surgeons whoperformed the surgical procedures, and so forth. In one example, atleast one of the plurality of surgical procedures are of the particulartype of surgical procedure described above in relation to 502. In oneexample, at least two but not all surgeries of the plurality of surgicalprocedures are of the particular type of surgical procedure describedabove in relation to 502. In another example, all surgeries of theplurality of surgical procedures are of the particular type of surgicalprocedure described above in relation to 502. The group of surgeons canrepresent surgeons who share a particular specialty. A medical specialtyis a branch of medical practice that is focused on a defined group ofpatients, diseases, skills, or philosophy. Examples include thosebranches of medicine that deal exclusively with children (paediatrics),cancer (oncology), laboratory medicine (pathology), or primary care(family medicine). The group of surgeons can belong to the same entityor can belong to separate entities. For example, the group of surgeonscan belong to the same medical institution, or to the same department inthe same medical institution.

The plurality of surgical procedures can be understood with reference toFIG. 7 . FIG. 7 is an illustration showing an example schedule that mayinclude a listing of procedures such as procedures A-C(e.g., surgicalprocedures, or any other suitable medical procedures that may beperformed in an operating room for which the schedule is used). For eachprocedure A-C, a corresponding starting and finishing times may bedetermined. For example, for a past procedure A, a starting time 1521Aand a finishing time 1521B may be the actual starting and finishingtimes. (Since procedure A is completed, the schedule may beautomatically updated to reflect actual times). For a current procedureB, a starting time 1523A may be actual and a finishing time 1523B may beestimated (and recorded as an estimated time). Additionally, forprocedure C, that is scheduled to be performed in the future, a startingtime 1525A and a finishing time 1525B may be estimated and recorded. Itshould be noted that the schedule is not limited to displaying and/orholding listings of procedures and starting/finishing times for theprocedures, but may include various other data of an example surgicalprocedure

Returning to FIG. 5 , in 506, a repository of intracorporeal videofootage is accessed. A repository may refer to any storage location orset of storage locations where video footage may be storedelectronically. For example, the repository may include a memory device,such as a hard drive and/or flash drive. In some aspects, the repositorymay be a network location such as a server, a cloud storage location, ashared network drive, or any other form of storage accessible over anetwork. The repository may include a database of surgical video footagecaptured at various times and/or locations. In some aspects, therepository may store additional data besides the surgical video footage.The repository may be a searchable repository, a sorted repository, anindexed repository, or any other repository as would be appreciated by aperson of ordinary skill in the art. The intracorporeal video footagemay depict performance of the plurality of surgical procedures by thegroup of surgeons. The repository of intracorporeal video footage can beunderstood with reference to FIG. 7 , described above.

At step 506, repository may be accessed to retrieve footage depicting atype of surgical procedure. For example, with reference to FIGS. 8A-8C,the intracorporeal video footage may depict performance of alaparoscopic cholecystectomy. The intracorporeal video footage depictperformance of a laparoscopic cholecystectomy by the group of surgeonsidentified in step 504. FIGS. 8A-8C are illustrations of intracorporealvideo streams, according to some aspects of the present disclosure.

Returning to FIG. 5 , in 508, for each surgical procedure of depicted inthe video retrieved in step 506, an image analysis is performed on theintracorporeal video footage. Respective intracorporeal video footagecan capture respective surgical procedures.

The image analysis is used to determine, at least, whether a respectiveaction from the set of actions specified in the particular surgicalguidelines occurred during performance of the respective surgicalprocedure. In some aspects, a visual action recognition algorithm may beused to analyze the intracorporeal video footage to determine whether arespective action occurred during a respective surgical procedure.Turning back to the example in FIG. 8A-C, FIG. 8A illustratesintracorporeal video stream 800, including action 1 802 and action 3804. FIG. 8B illustrates intracorporeal video stream 820, includingaction 2 822 and action 3 824. FIG. 8C illustrates intracorporeal videostream 840, including action 1 842.

To conduct the image analysis, a machine learning model may be trainedusing training examples to analyze surgical images and/or videos todetermine whether particular actions occurred. For example, a samplesurgical image and/or a sample surgical video of a sample surgicalprocedure, together with a label indicating whether a sample actionoccurred in the sample surgical procedure, can be used as a trainingexample. The trained machine learning model may be used to analyze theintracorporeal video footage to determine whether the respective actionoccurred during the respective surgical procedure. The machine learningmodel can also be trained to determine other metrics.

To conduct the image analysis, a convolution of at least part of theintracorporeal video footage may be calculated to obtain a result valueof the calculated convolution. Further, determining whether therespective action occurred in the respective surgical procedure may bebased on the result value of the calculated convolution of at least partof the intracorporeal video footage. For example, when the result valueis a first numerical value, it may be determined that the respectiveaction occurred in the respective surgical procedure, and when theresult value is a second numerical value, it may be determined that therespective action did not occur in the respective surgical procedure.

In an aspect, the set of actions can include a first action and a secondaction. In this aspect, the image analysis can determine whether thefirst action occurred. Another analysis can then analyze other data,such as non-video data, to determine whether the second action occurred.In another example, analyzing both the image data and the non-videodata, for example using a multimodal artificial neural network, maydetermine whether a particular action included in the set of actionsoccurred. Accordingly, determining the aggregated compliance level canbe based on performing the image analysis and performing an analysis ofthe non-video data. In some examples, the non-video data may includeaudio data captured during the surgical procedures. Further, the audiodata may be analyzed (for example, using pattern recognition algorithms,using speech recognition algorithms, using speaker diarisationalgorithms, etc.) to determine whether the second action occurred. Forexample, the action may include one medical practitioner informinganother medical practitioner of an event (such as an event thatoccurred, an action that is about to take place, and so forth). In someother examples, the non-video data may include sensor data capturedusing a sensor included in a medical appliance, such as blood pressure,body temperature, friction, surface tension, electrical impedance,electrical flow, electrical resistance, electrical capacity, and soforth. Further, the sensor data may be analyzed (for example, using apattern recognition algorithm, using an artificial neural network, etc.)to determine whether the second action occurred. In some other examples,the non-video data may include textual data (such as medical records ofpatients undergoing the surgeries, postoperative reports, and so forth).Further, the textual data may be analyzed (for example, using NaturalLanguage Processing (NLP) algorithms, using artificial neural networks,etc.) to determine whether the second action occurred. In some otherexamples, the non-video data may include structured data (for example,from a database, from structured text files, from a data-structure, andso forth). Further, the structured data may be analyzed (for example, byencoding information from fields of the structured data in a vector, andanalyzing the vector using a machine learning model) to determinewhether the second action occurred.

The second action may include an action that typically occurs during asurgical procedure but is not typically captured in an intracorporealvideo footage, such as actions occurring extracorporeal (e.g., actionsrelated to anesthesia, transfusions, administration of drugs, surgicalcounting, etc.) or actions occurring intracorporeal but outside thefield of view of the intracorporeal video footage. In another aspect,the second action may include an action that typically occurs before asurgical procedure, such as a surgeon preparing for a surgery (e.g.,reviewing patient information, refreshing on a surgical technique,etc.), preparing the patient (e.g., administrating drugs, performingmedical tests, etc.), and so forth. In another aspect, the second actionmay include an action that typically occurs after a surgical procedure,such as administrating drugs, conducting medical tests, and so forth.

In 510, based on the image analysis from 508, an aggregated compliancelevel is determined for the group of surgeons. The aggregated compliancelevel indicates a degree to which the group of surgeons adhered to theparticular surgical guideline in performing the plurality of surgicalprocedures. In one example, in 510, based on the determinations ofwhether the respective action from the set of actions specified in theparticular surgical guideline occurred in the different surgicalprocedures, the aggregated compliance level of the group of surgeons maybe determined.

In some aspects, an aggregated compliance level may be determined basedon a statistical function of the compliance levels determined for eachsurgical procedure of the plurality of surgical procedures. Thecompliance level of a particular surgical procedure may be a numericalvalue, a continuous numerical value, a discrete numerical value, adiscrete grade (e.g., “Good,” “Med.,” and “Low”), and so forth. Anotherexample of such aggregated compliance level may include a ratio ofoccurred actions of a particular surgical procedure for a total set ofactions specified in a particular surgical guideline. In one example,the particular surgical guideline may further indicate, for each action,a level indicative of the importance and/or criticality of therespective action, for example as described above in relation to 502.Further, the ratio of occurred actions of a particular surgicalprocedure for a total set of actions specified in a particular surgicalguideline may be a weighted ratio with weights selected based on thelevels corresponding to the actions, for example giving higher weight tomore important and/or critical actions.

In some aspects, a machine learning model may be trained using trainingexamples to determine a compliance level based on whether actionsoccurred. An aspect of such training example may include informationindicative of whether sample actions occurred in a sample surgicalprocedure, together with a label indicative of a sample compliance levelrelated to the sample surgical procedure. The trained machine learningmodel may be used to determine the compliance level for a particularsurgical procedure based on whether actions of a set of actionsspecified in a particular surgical guideline occurred in the particularsurgical procedure.

In some aspects, a statistical measure of the determined compliancelevels may be calculated to obtain the aggregated compliance level ofthe group of surgeons. Some non-limiting examples of such statisticalmeasure may include mean, median, mode, or any other statisticalfunction.

In aspects where the data structure identifies a plurality of surgicalguidelines, the aggregated compliance level can indicate an overallcompliance level of the group of surgeons to each surgical guideline inthe plurality of surgical guidelines. Additionally, based on the imageanalysis from 508, an individual compliance level of a respectivesurgeon from the group of surgeons can be determined. The individualcompliance level can indicate a degree to which the respective surgeonadhered to the particular surgical guideline in performing the pluralityof surgical procedures. For example, the individual compliance level maybe determined as described above in relation to the aggregatedcompliance level, when applied for a group of surgeons that includesonly the respective surgeon.

In some aspects, the set of actions of 502 may include at least oneconditional action to be performed only when a specified situation isencountered. For example, in laparoscopic cholecystectomy, a conditionalaction may include performing a total cholecystectomy only in caseswhere a Critical View of Safety is reached. In another example, inlaparoscopic cholecystectomy, a conditional action may includeperforming subtotal cholecystectomy in cases in which the surgeon isunable to achieve a Critical View of Safety. The image analysis of 508may further determine whether a condition associated with a conditionalaction was satisfied. For example, the condition may include asuccessful completion of an action. Further, 510 may increase theaggregated compliance level when the action of a conditional action isperformed in surgical procedures where the condition of the conditionalaction is satisfied. Further, 510 may decrease the aggregated compliancelevel when the action of a conditional action is performed in surgicalprocedures where the condition of the conditional action is notsatisfied. Further, 510 may decrease the aggregated compliance levelwhen the action of a conditional action is not performed in surgicalprocedures where the condition of the conditional action is satisfied.

In some aspects, the aggregated compliance level can be determined basedin part on a non-video data source. For example, the non-video datasource can be data indicating use of a software application, such as asoftware application designed to guide the surgeon through a checklistfor the type of surgery.

In 512, an indicator of the aggregated compliance level is output. Forexample, the indicator may be stored in memory, may be transmitted to anexternal computing device, may be presented to an individual (e.g., viaa user interface, visually, audibly, textually, graphically, etc.), andso forth. An additional indicator can be output when compliance levelsare determined for respective surgeons. The additional indicator canindicate the individual compliance level of the respective surgeon. Forexample, the indicator and/or the additional indicator may be providedto an individual associated with the group of surgeons, such as adepartment head, a quality leader, an insurer, and so forth.

In some aspects of process 500, the group of surgeons can be determinedas surgeons of a medical institution. Accordingly, the plurality ofsurgical procedures can be determined as surgeries of a particular typetypically performed by the group of surgeons. Records describing theplurality of surgical procedures, such as the intracorporeal videofootage, can then be accessed. It is also possible for a second datastructure to be accessed. The second data structure can containuniversal aggregate data for the particular type of surgery. Then, basedon the aggregated compliance level and the universal aggregate data forthe particular type of surgery, an institutional deficiency can beidentified in a performance factor describing performance (e.g., safetyand efficacy) of the particular type of surgery. In this way, asuggested surgical guideline for addressing the institutional deficiencycan be provided based on the deficiency and the particular type ofsurgery. In some aspects, the suggested surgical guidelines can also bebased on existing surgical guidelines that the medical institutionfollows. The suggested surgical guidelines can include a referencemedical institution that currently follows the suggested surgicalguidelines. Some non-limiting examples of such performance factor mayinclude durations of surgeries, outcomes, readmission rates, costs, andso forth. In some examples, a machine learning model may be trainedusing training examples to suggest surgical guidelines based oninstitutional deficiencies and/or based on surgical guidelines that arecurrently followed and/or baed on records associated with surgeries. Anexample of such training example may include a sample institutionaldeficiency and/or a sample surgical guideline that is currently followedand/or a sample record associated with a sample surgery, together with alabel indicative of a suggested surgical guideline for addressing thesample institutional deficiency. The trained machine learning model maybe used to analyze the institutional deficiency and/or other informationto determine the suggested surgical guidelines.

Where the group of surgeons belongs to the same entity (such as amedical institution), recommendations and corrective actions for theentity may be determined. In that embodiment, image analysis isperformed before and after a recommendation is provided, for example, todetermine an improvement as result of the recommendation. First, imageanalysis is performed on the intracorporeal video footage capturing therespective surgical procedure to detect a first set of intraoperativeactions performed during the particular type of surgical procedure.Based on the detected first set of intraoperative actions, a series ofvideo frames can be determined. The determined series of frames depict afailure to comply with a surgical guideline during the particular typeof surgical procedure. For example, the intraoperative actions may bedetected as described above, for example using a visual actionrecognition algorithm or a trained machine learning model. In oneexample, the series of video frames may depict a failure to comply witha surgical guideline by performing a first action before a successfulcompletion of the second action. In this example, the determined seriesof video frames may include depiction of the first action and the secondaction showing the first action taking place before the second action.In another example, the series of video frames may depict a failure tocomply with a surgical guideline by performing a particular action whenit should be avoided based on the status of the patient or the surgery.In this example, the determined series of video frames may includedepiction of the particular action. In some examples, a machine learningmodel may be trained to select frames of a surgical video that depicts afailure to comply with a surgical guideline based on actions detected inthe surgical video. An example of such training example may include asample surgical video of a sample surgery and a sample set of actionsperformed in the sample surgery, together with a label indicating asample selection of frames of the sample surgical video that depicts afailure to comply with a particular surgical guideline. The trainedmachine learning model may be used to analyze the first set ofintraoperative actions to determine the series of video frames. Based onthe determined series of video frames, a suggested surgical guidelinerecommendation can be provided to the entity. In some examples, amachine learning model may be trained using training examples to suggestsurgical guidelines based on frames of surgical videos. An example ofsuch training example may include sample selected frames of a samplesurgical video, together with a label indicating a suggested surgicalguideline. The trained machine learning model may be used to analyze thedetermined series of video frames to select a suggested surgicalguideline, and a recommendation for the entity to follow the suggestedsurgical guideline may be provided. In some examples, the providedrecommendation for the entity to follow the suggested surgical guidelinemay include the determined series of video frames. In some examples, theprovided recommendation for the entity to follow the suggested surgicalguideline may include information determined based on an analysis of thedetermined series of video frames.

Then, an image analysis can be performed on a second intracorporealvideo footage that is captured after the recommendation. The secondintracorporeal video footage depicts performance of the same particulartype of surgical procedure as was analyzed before the recommendation.This second image analysis can be used to detect a second set ofintraoperative actions. This image analysis can be conducted asdescribed above in relation to the first set of intraoperative actionsand the intracorporeal video footage capturing the respective surgicalprocedure. Based on the detected second set of intraoperative actions,and/or on an analysis of the second intracorporeal video, a subsequentcompliance level, with the specific surgical guideline, can bedetermined. The subsequent compliance level can be determined asdescribed above for step 510.

This subsequent compliance level can be used to initiate a correctiveaction. The corrective action can be selected such that it is based onmore than one subsequent recommendation. In some aspects, the correctiveaction can include providing a reference video stream demonstratingproper adherence to the selected surgical guideline. In some aspects,the corrective action can include automatic adjustment of an insurancepremium of an associated insurance policy. In some aspects, thecorrective action can include automatically notifying a supervisor ofthe group of surgeons or the entity. In some aspects, the correctiveaction can include providing a reminder to the entity, to remind thegroup of surgeons to follow the specific surgical guideline during apreparation for a future surgical procedure.

In some aspects of process 500, the plurality of surgical procedures canbe divided into subgroups. Then, for each surgical procedure of aparticular subgroups, image analysis can be performed on a particularintracorporeal video footage from the repository, where the videofootage captures a particular surgical procedure. This analysis candetermine a failure type describing a failure to perform a particularaction in the particular surgical procedure (continuing the runningexample, such as a failure to use a Critical View of Safety during alaparoscopic cholecystectomy), where the particular action is includedin the set of actions specified in the particular surgical guideline.Using the determined failure types, a statistical measure can bedetermined, and an additional indicator indicative of the measure can beoutput. The statistical measure describing occurrence the determinedfailure types among the identified group of surgeons. In an example, thestatistical measure can be a frequency or percentage of times thefailure types at a particular medical institution. In some aspects, thefailure type can be not attempting an action or can be attempting butnot successfully completing the action.

In this way, using the techniques described above with respect to method500 in FIG. 5 , medical centers or departments can determining a degreeto which surgeons having a particular specialty comply with surgicalguidelines.

In some examples, a data structure identifying a particular surgicalguideline specifying a set of actions to take place during alaparoscopic cholecystectomy may be accessed. For example, the surgicalguideline illustrated by FIG. 6 may be accessed as described in relationto 502. For example, the set of actions may include ‘Use of CriticalView of Safety (CVS) technique’ (Action 1), ‘If the CVS is notachievable, perform a bailout procedure such as subtotalcholecystectomy’ (Action 2), and ‘Use of at least one of intraoperativecholangiography or laparoscopic ultrasound’ (Action 3). Further, a groupof surgeons who performed a plurality of surgical procedures may beidentified, for example as described in relation to 504. In one example,at least two but not all surgeries of the plurality of surgicalprocedures are laparoscopic cholecystectomy. In another example, allsurgeries of the plurality of surgical procedures are laparoscopiccholecystectomy. Further, a repository of intracorporeal video footagedepicting performance of the plurality of surgical procedures by thegroup of surgeons may be accessed, for example as described above inrelation to 506. Further, for each laparoscopic cholecystectomy of theplurality of surgical procedures, image analysis may be performed onintracorporeal video footage from the repository, where theintracorporeal video footage may capture the respective laparoscopiccholecystectomy. The image analysis may be used to determine at leastwhether a respective action from the set of actions specified in theparticular surgical guideline occurred in the respective laparoscopiccholecystectomy, for example as described above in relation to 508.Further, based on the image analysis and/or based on the determinationsof whether the respective action from the set of actions specified inthe particular surgical guideline occurred in the different laparoscopiccholecystectomies, an aggregated compliance level of the group ofsurgeons may be determined, for example as described above in relationto 510. The aggregated compliance level may indicate a degree to whichthe group of surgeons adhered to the particular surgical guideline inperforming the laparoscopic cholecystectomies. Further, an indicator ofthe aggregated compliance level may be outputted, for example asdescribed above in relation to Step 512.

Medical centers or departments may wish to check compliance with acustomized set of guidelines chosen from a larger group of guidelines.For example, if a hospital has unusually high post-operative infectionrates, guidelines that impact infection may be selected for analysis.Once selected, the surgical video streams of one or more surgicalprocedures may be compared with the guidelines. In some aspects, anon-transitory computer readable medium can contain instructions that,when executed by at least one processor, cause the at least oneprocessor to execute operations to perform intracorporeal video analysisoperations for monitoring compliance with surgical guidelines.

FIG. 9 is a flowchart of an example process 900 for determiningcompliance with selected surgical guidelines, according to some aspectsof the present disclosure. It is to be appreciated that not all stepscan be needed to perform the disclosure provided herein. Further, someof the steps can be performed simultaneously, or in a different orderthan shown in FIG. 9 , as will be understood by a person of ordinaryskill in the art.

Process 900 can be implemented by devices, systems, and operationsdescribed in FIGS. 1-8 and using operations caused by computer system2000. Process 900 can also be understood with reference to FIG. 10 .However, process 900 is not limited to these example aspects.

In 902, descriptors of a group of surgical guidelines is presented. Inone example, the descriptors and/or the group of surgical guidelines maybe read from memory, may be received from an external computing device(for example, using a digital communication device), may be receivedfrom an individual (for example, via a user interface), may be generated(for example, using a generative machine learning model), and so forth.

A category for each surgical guideline can be presented as at least onedescriptor. An example of such category may include importance (such as“Mandatory,” “Highly Recommended,” “Recommended,” and so forth). Inanother example, the category may relate to an entity related to theguideline (such as an entity enforcing, suggesting, or supporting theguideline). Some other non-limiting examples of such category mayinclude “Safety,” “Quality,” “Efficiency,” and so forth.

Links to medical articles supporting the group of surgical guidelinescan be presented as at least one descriptor. Another descriptor caninclude a predicted effect to comply with for each surgical guideline.For example, the predicted effect can be an intraoperative outcome or apostoperative outcome.

The descriptors may be presented to an individual, may be presentedvisually, may be presented audibly, may be presented textually, may bepresented graphically, or may be presented via a user interface.

FIG. 10 is an illustration of groups of surgical guidelines, includingdescriptors, according to some aspects of the present disclosure. Forexample, surgical guidelines can be divided into a “Preoperative” group,a “Surgical events” group, and a “Postoperative” group. In addition, theguidelines include descriptors indicating whether the respectiveguideline reduces risk of infection.

Returning to FIG. 9 , in 904, a selection of a subgroup of surgicalguidelines from the group of surgical guidelines is received. Thesubgroup can include at least one, but not all, of the surgicalguidelines from the group of surgical guidelines. The selection may bereceived from an individual, may be read from a memory, may be receivedfrom an external computing device (e.g., using a digital communicationdevice), or may be received via a user interface. In another example,the group of surgical guidelines may be analyzed to select the subgroupof surgical guidelines from the group of surgical guidelines. Forexample, a machine learning model may be trained using training examplesto select a subset of guidelines for a medical entity based on dataassociated with the medical entity. An example of such training examplemay include sample data associated with a sample medical entity,together with a label indicating a sample selection of a sample subsetof a sample plurality of guidelines. The trained machine learning modelmay be used to select the subgroup of surgical guidelines from the groupof surgical guidelines, for example based on data associated with aparticular medical entity. Some non-limiting examples of such dataassociated with the medical entity may include size of the medicalentity, number of surgeries (of a particular type) performed by themedical entity in a selected time period, number of surgeons affiliatedwith the medical entity, complications in past surgeries performed bythe medical entity, medical insurance associated with the medicalentity, level of experience of surgeons associated with the medicalentity, and so forth. Some non-limiting examples of such medical entitymay include a medical center, a department, a group of surgeons workingtogether, and so forth.

As shown in FIG. 10 , the groups of surgical procedures can also includea subgroup of surgical guidelines, such as those surgical guidelinesaimed at reducing risk of infection. For example, the guidelines of“Preoperative antibiotics in high-risk patients[,]” “Use of criticalview of safety (CVS) technique[,]” and “Clipping bile duct[,]” can beselected as a subgroup of surgical guidelines, related to reducing riskof infection, even though they fall under different groups ofguidelines. Note that this subgroup of surgical guidelines does not needto include a surgical guideline from all groups (e.g., no guidelinesfrom “Postoperative” are included in the subgroup), and does not need toinclude all surgical guidelines from a particular group (e.g., someguidelines from “Surgical events” are excluded from the subgroup).

Returning to FIG. 9 , in 906, a repository of intracorporeal videostreams can be accessed. A repository may refer to any storage locationor set of storage locations where video footage may be storedelectronically. For example, the repository may include a memory device,such as a hard drive and/or flash drive. In another example, therepository may be controlled in one or more external computing devices,and accessing the repository may include communicating with the one ormore external computing devices using a digital communication device. Insome aspects, the repository may be a network location such as a server,a cloud storage location, a shared network drive, or any other form ofstorage accessible over a network. The repository may include a databaseof surgical video footage captured at various times and/or locations. Insome aspects, the repository may store additional data besides thesurgical video footage. The repository may be a searchable repository, asorted repository, an indexed repository, or any other repository aswould be appreciated by a person of ordinary skill in the art. The videostreams can depict performance, by at least one surgeon, of a pluralityof surgical procedures governed by the subgroup of surgical guidelines.

In some aspects, the at least one surgeon can include a plurality ofsurgeons. In such aspects, 908 and 910 can be repeated for each of theplurality of surgeons.

In 908, for each surgical procedure of the plurality of surgicalprocedures, an image analysis can be performed on a particularintracorporeal video stream from the repository. The particularintracorporeal video stream can capture a respective surgical procedure.The image analysis can be used to determine whether at least onesurgical guideline was followed or complied with. Such image analysiswas described above with respect to step 508 in FIG. 5 .

In 910, based on the image analysis in 908, an aggregated compliancelevel can be determined. The aggregated compliance level can indicatecompliance, of the at least one surgeon, with the selected subgroup ofsurgical guidelines. The aggregated compliance level can be determinedusing a variety of formulas and statistical techniques as was describedabove with respect to step 510 in FIG. 5 .

In some aspects, the plurality of surgical procedures were performedover a period of time, such that the aggregated compliance levelreflects an overall compliance of the subgroup of surgical guidelinesover that period of time.

In some aspects, the aggregated compliance level can be compared to athreshold to determine whether the surgeon is in violation of policiesor guidelines. For example, when an aggregated compliance level is lessthan a threshold, one or more recommendations for improving complianceof a specific surgical guideline from the subgroup of surgicalguidelines can be presented. In one example, the recommendations may beselected as described above.

The subgroup of surgical guidelines can include both guidelines that arenot video associated and those that are. For example, turning to FIG. 10, “Preoperative antibiotics” and “Control of postoperative pain”describe actions conducted in support of the surgery that may not becaptured in surgical video, while “Use of critical view of safety (CVS)technique” and “Clipping bile duct” may be captured in surgical video.In other examples, the guidelines that are not video associated mayinclude actions conducted while a surgery is ongoing, for example asdescribed above.

In such aspects, the image analysis is performed on the intracorporealvideo stream to determine compliance with the video-associated medicalguideline. Additionally, at least one record of at least onenon-surgical action to support the respective surgical procedure can beretrieved. The at least one non-surgical action can include an actionperformed before, during or after the respective surgical procedure. Theat least one non-surgical action can include at least one of a recordkeeping action, usage of a monitoring device, preparation, or debrief.Other non-limiting examples of such non-surgical action are describedabove. This at least one record can be analyzed to determine compliancewith the second non-video-associated medical guideline, for example asdescribed above. In this aspect, the aggregated compliance level isdetermined based on the determined compliance with the firstvideo-associated guideline and the determined compliance with the secondnon-video-associated guideline.

In 912, an indicator of the aggregated compliance level can be output.For example, the indicator may be stored in memory, may be transmittedto an external computing device, may be presented to an individual(e.g., via a user interface, visually, audibly, textually, graphically,etc.), and so forth. For example, the indicator may be provided to anindividual associated with the at least one surgeon, such the at leastone surgeon, a manager of the at least one surgeon, a department head, aquality leader, an insurer, and so forth.

In aspects where the subgroup of surgical guidelines includes bothguidelines associated with video and those that are not, the indicatorcan include a first indicator reflecting a compliance level to the firstvideo-associated medical guideline and a second indicator reflecting acompliance level to the non-video-associated medical guideline.

In some aspects of process 900, an image analysis can be performed on aparticular intracorporeal video stream from the repository to detectintraoperative actions performed during a particular surgical procedure,for example as described above in relation to process 500. First, imageanalysis may be performed on the intracorporeal video footage capturingthe respective surgical procedure to detect a first set ofintraoperative actions performed during the particular surgicalprocedure. Based on the detected first set of intraoperative actions, aseries of video frames can be determined, for example as describe above.The determined series of frames may depict a failure to comply with aparticular surgical guideline during the particular surgical procedure.The series of video frames can include at least one detectedintraoperative action that took place prior to the failure to complywith the particular surgical guideline. Based on the video frames, asuggested surgical guideline recommendation can be provided to anentity. This determination may be made as described above, for exampleusing a machine learning model. The recommendation may be selectedand/or generated as described above in relation to process 500.

Then, an image analysis can be performed on a second intracorporealvideo footage that is captured after the recommendation. The secondintracorporeal video footage depicts performance of the same particulartype of surgical procedure as was analyzed before the recommendation.This second image analysis can be used to detect a second set ofintraoperative actions. This image analysis can be conducted asdescribed above. Based on the detected second set of intraoperativeactions, a subsequent compliance level, with the specific surgicalguideline, can be determined. The subsequent compliance level can bedetermined as described above. This subsequent compliance level can beused to initiate a corrective action. In some examples, the correctiveaction may be selected of a plurality of alternative corrective actions,for example based on the recommendation and the subsequent compliancelevel. In some examples, a textual content aimed to cause an individualto initiate the corrective action may be generated, for example using aLarge Language Model (LLM) or other generative models, for example basedon the recommendation and the subsequent compliance level. In someexamples, a machine learning model may be trained using trainingexamples to select corrective actions based on recommendations andsubsequent compliance levels. An example of such training example mayinclude a sample recommendation and a sample subsequent compliancelevel, together with a label indicating a sample corrective action. Thetrained machine learning model may be used to analyze the recommendationand the subsequent compliance level to select the corrective action tobe initiated. In some examples, corrective action can be selected suchthat it is based on more than one subsequent recommendation. In someaspects, the corrective action can include providing a reference videostream demonstrating proper adherence to the selected surgicalguideline. In some aspects, the corrective action can include automaticadjustment of an insurance premium of an associated insurance policy. Insome aspects, the corrective action can include automatically notifyinga supervisor of surgeons or the entity. In some aspects, the correctiveaction can include providing a reminder to the entity, to remind thesurgeons to follow the specific surgical guideline during a preparationfor a future surgical procedure.

As mentioned above, a specific reference video stream demonstratingadherence to the particular surgical guideline can be selected from therepository. The specific reference video stream can be selected based onone or more characteristics of the particular surgical procedure. Forexample, the one or more characteristics can be determined as duration,duration of a particular step, complexity level, association with aspecific anatomical abnormality, characteristics of the patientundergoing the surgical procedure (e.g., age, gender, blood type,anatomical characteristics, medical condition, etc.), characteristics ofthe surgeon performing the surgical procedure, and so forth. Additionalintracorporeal video streams can also be analyzed to identify othersurgical procedures in which the particular surgical guideline was notcomplied with.

In such aspects of process 900, possible or likely reasons for thefailure to comply with the particular surgical guideline can bedetermined. The failure can be analyzed in view of the other surgicalprocedures. In some aspects, one or more commonalities among theparticular surgical procedure and the other surgical procedures may beidentified. For example, a particular action preceding the failure, acommon characteristic of the surgical procedures, and so forth, may beidentified. Commonalities may be determined, for example, using machinelearning, RANdom Sample Consensus (RANSAC) and/or clustering algorithms(such as K-means clustering). The prevalence of each commonality of theone or more commonalities in a second group of surgical procedures inwhich the particular surgical guideline was complied with may bedetermined. Any commonality having a low prevalence in the second groupmay be determined to be a likely reason for the failure.

In other aspects, a plurality of potential reasons may be examined todetermine the likely reason for the failure. For example, theintracorporeal video streams of the particular surgical procedure and/orthe other surgical procedures may be analyzed to determine whether aparticular potential reason is a likely reason for the failure. Forexample, a machine learning model may be trained using training examplesto determine whether a particular potential reason is a likely reasonfor a failure based on surgical images and/or videos. An example of suchtraining example may include a sample surgical image and/or a samplesurgical video of a sample surgical procedure, together with a labelindicating whether the particular potential reason is a likely reasonfor a sample failure in the sample surgical procedure. The trainedmachine learning model may be used to analyze the intracorporeal videostreams of the particular surgical procedure and/or the other surgicalprocedures to determine whether a particular potential reason is alikely reason for the failure.

Then, the series of video frames depicting the failure to comply withthe particular surgical guideline and the specific reference videostream demonstrating adherence to the particular surgical guideline canbe output. The series of video frames may be identified by using theimage analysis described above to determine when the relevant actionsoccurred in the video. These video frames and video stream can bepresented to a user, via a user interface. The user interface cansimultaneously present the outputted video frames and video stream. Insome aspects, the simultaneous presentation can be in an aligned timingsequence. In aspects where possible or likely reasons for failure aredetermined, an indication of the reasons can also be output.

In aspects where the user interface is simultaneously presenting theoutputted video frames and video stream, the user interface can beconfigured to modify at least one of the output series of video framesand the specific reference video stream to match one or more displayparameters. For example, at least one of the series of video frames andthe specific reference video stream may be transformed so that the twooutputs appear substantially from the same angle. In another example, atleast one of the series of video frames and the specific reference videostream may be transformed so that the same anatomical structure appearsin the same region of the two outputs and/or in the same scale.

In this way, using the techniques descried with respect to FIG. 9 ,medical centers can check compliance with a customized set of guidelineschosen from a larger group of guidelines.

In some examples, descriptors of a group of surgical guidelines may bepresented, for example as illustrated in FIG. 10 . Further, a selectionof a subgroup of surgical guidelines from the group of surgicalguidelines may be received, for example from an individual via a userinterface. The subgroup including at least one but not all of thesurgical guidelines from the group of surgical guidelines. For example,the subgroup may include ‘Use of critical view of safety (CVS)technique’ and ‘Clipping bile duct’. Further, a repository ofintracorporeal video streams depicting performance, by at least onesurgeon, of a plurality of surgical procedures governed by the subgroupof surgical guidelines may be accessed. For example, the intracorporealvideo streams may depict performance of laparoscopic cholecystectomy bya particular surgeon. Further, for each surgical procedure of theplurality of surgical procedures, image analysis may be performed on anintracorporeal video stream capturing the respective surgical procedurefrom the repository, for example as described in relation to 908. Theimage analysis may be used to determine whether at least one surgicalguideline was completed. Further, based on the image analysis, anaggregated compliance level indicative of compliance of the particularsurgeon with the selected subgroup of surgical guidelines may bedetermined, for example as described above in relation to 910. Further,an indicator of the aggregated compliance level may be provided, forexample to the individual, to the particular surgeon, to a third entity,and so forth.

In an embodiment, physician, medical department or medical instituteinsurance premiums may be dynamically adjusted based on the level ofcompliance with surgical guidelines determined through video analysis.This dynamic adjustment can improve reliability and performance of thedistributed computer system by reducing the number of computinginteractions needed to determine a new insurance premium.

FIG. 11 is a flowchart of an example process 1100 of surgical videoanalysis for insurance premium adjustment, according to some aspects ofthe present disclosure. It is to be appreciated that not all steps canbe needed to perform the disclosure provided herein. Further, some ofthe steps can be performed simultaneously, or in a different order thanshown in FIG. 11 , as will be understood by a person of ordinary skillin the art.

Process 1100 can be implemented by devices, systems, and operationsdescribed in FIGS. 1-10 and using operations caused by computer system2000. However, process 1100 is not limited to these example aspects.

In 1102, a data structure identifying a plurality of surgical guidelinescan be accessed. Each surgical guideline of the plurality of surgicalguidelines can specify a set of actions to take place during a type ofsurgical procedure.

The plurality of surgical guidelines can be received, for example, via aselection. For example, the selection may be received from anindividual, may be read from a memory, may be received from an externalcomputing device (e.g., using a digital communication device), or may bereceived via a user interface. In another example, a group ofalternative surgical guidelines and/or data associated with a medicalentity may be analyzed to select the plurality of surgical guidelinesfrom the group of alternative surgical guidelines. For example, amachine learning model may be trained using training examples to selecta subset of guidelines for a medical entity based on data associatedwith the medical entity. An example of such training example may includesample data associated with a sample medical entity and/or a samplegroup of alternative guidelines, together with a label indicating asample selection of a sample subset of a sample plurality of guidelinesof the sample group of alternative guidelines. The trained machinelearning model may be used to select the plurality of surgicalguidelines, for example based on data associated with a selected surgeon(such as the surgeon selected in 1104) and/or data associated with aninsurance policy (such as the insurance policy of 1104) and/or a groupof alternative guidelines.

In 1104, a selection of a surgeon covered by an original insurancepolicy can be received. For example, the selection may be received froman individual, may be read from a memory, may be received from anexternal computing device (e.g., using a digital communication device),or may be received via a user interface.

In 1106, a repository of a plurality of intracorporeal video streams canbe accessed. A repository may refer to any storage location or set ofstorage locations where video footage may be stored electronically. Forexample, the repository may include a memory device, such as a harddrive and/or flash drive. In some aspects, the repository may be anetwork location such as a server, a cloud storage location, a sharednetwork drive, or any other form of storage accessible over a network.The repository may include a database of surgical video footage capturedat various times and/or locations. In some aspects, the repository maystore additional data besides the surgical video footage. The repositorymay be a searchable repository, a sorted repository, an indexedrepository, or any other repository as would be appreciated by a personof ordinary skill in the art. Each intracorporeal video stream candepict a surgical procedure performed by the surgeon.

In 1108, for each of the plurality of intracorporeal video streams, animage analysis can be performed on the respective intracorporeal videostream, for example as described above in relation to 508 and/or 908.The image analysis can be used to determine whether at least one actionfrom the set of actions specified in a surgical guideline occurred. Thesurgical guideline can govern the surgical procedure depicted in therespective intracorporeal video stream.

In some aspects, for a particular intracorporeal video stream of theplurality of intracorporeal video streams, a convolution of at leastpart of the particular intracorporeal video stream can be calculated toobtain a result value. Further, the result value can then be used todetermine whether the at least one action from the set of actionsoccurred. For example, when the result value is a first numerical value,it may be determined that a particular action occurred, and when theresult value is a second numerical value, it may be determined that theparticular action didn't occurred.

In some aspects, for a particular intracorporeal video stream of theplurality of intracorporeal video streams, a machine learning model canbe used to analyze the particular intracorporeal video stream. Themachine learning model can be used to determine whether the at least oneaction from the set of actions occurred. For example, the machinelearning model may be a machine learning model trained using trainingexamples to determine whether actions occur from intracorporeal videostreams. An example of such training example may include a sampleintracorporeal video stream, together with a label indicating whether asample action occurred in the sample intracorporeal video stream.

In 1110, based on the image analysis in 1108, a level of complianceindicating a degree to which the surgeon complied with at least some ofthe plurality of surgical guidelines can be determined, for example asdescribed above with respect to process 500 and process 900.

The plurality of intracorporeal video streams can depict a plurality ofsurgical procedures performed during a time period, such that the levelof compliance reflects an adherence to the plurality of surgicalguidelines during that time period. Additionally, a data structureincluding a reference compliance level can be accessed. This referencecompliance level can reflect an adherence to the plurality of surgicalguidelines during a previous time period. Further, the adjustment to theinsurance premium may be based on a difference between the aggregatedcompliance level and the reference compliance level. For example, anamount of the adjustment to the insurance premium may be a function ofthe difference between the aggregated compliance level and the referencecompliance level. Some non-limiting examples of such function mayinclude a linear function, a non-linear function, a polynomial function,an exponential function, a continuous function, a discontinuousfunction, and so forth. In another example, a limitation may be added toand/or removed from the insurance policy based on the difference betweenthe aggregated compliance level and the reference compliance level.

The original insurance policy can cover a plurality of surgeons. In thisaspect, 1106-1110 can be repeated for each of the plurality of surgeons.Accordingly, an aggregated compliance level can be determined based onthe level of compliance for each of the plurality of surgeons.

In 1112, information based on the level of compliance can be output.This output can enable a determination of an adjustment to an insurancepremium of the original insurance policy. For example, outputting theinformation based on the level of compliance may include presenting theinformation based on the level of compliance to an individual to enablethe individual to adjust the insurance premium, for example via a userinterface. In another example, outputting the information based on thelevel of compliance may include transmitting a digital signal encodingthe information based on the level of compliance to an externalcomputing device to enable the external computing device to adjust theinsurance premium, for example using a digital communication device. Inyet another example, outputting the information based on the level ofcompliance may include storing the information based on the level ofcompliance in a digital memory to enable other software processes and/oralgorithms to access the information based on the level of complianceand adjust the insurance premium. The adjustment to the insurancepremium can be by an amount that is based on the determined level ofcompliance of the surgeon. For example, the adjustment to the insurancepremium can be by an amount that is a function of the determined levelof compliance of the surgeon. Some non-limiting examples of suchfunction may include a linear function, a non-linear function, apolynomial function, an exponential function, a continuous function, adiscontinuous function, and so forth. An indication of the adjustmentcan also be provided as an output. A condition of the adjustment canalso be determined based on the plurality of intracorporeal videostreams and an indication of the condition can be output. An example ofthe adjustment may be, for instance, that the adjustment may be valid aslong as the surgeon works no longer than 11 hours a day or does notperform surgeries of a particular type, and so forth. For example, whenthe plurality of intracorporeal video streams shows the surgeon is athigher risk when using a specific surgical technique, the condition maybe that the surgeon does not use the specific surgical technique. In oneexample, a machine learning model may be trained using training examplesto determine conditions for an insurance policy based on intracorporealvideo streams. An example of such training example, may include sampleintracorporeal video streams, together with a label indicating a sampleselection of conditions for a sample insurance policy. The trainedmachine learning model may be used to analyze the plurality ofintracorporeal video streams to determine the condition of theadjustment.

The adjustment to the insurance premium can be based on the determinedlevel of compliance and one or more other metrics. For example, the oneor more other metrics can include complexity levels of the surgicalprocedures depicted in the plurality of intracorporeal video streamsand/or characteristics of a patient undergoing a surgical procedure(e.g., weight, age, mobility, medical conditions, etc.) depicted in oneof the plurality of intracorporeal video streams. In another example,the one or more other metrics can include characteristics of a medicalequipment used in a surgical procedure (e.g., use of a correct orincorrect type of equipment, use of a specific equipment configuration,use of a specific equipment attachment, etc.) depicted in one of theplurality of intracorporeal video streams or on practitioners other thanthe selected surgeon performing or otherwise involved with the surgicalprocedures depicted in the plurality of intracorporeal video streams. Inyet another example, the one or more other metrics can be a temporaltrend. The temporal trend indicates the degree to which the surgeon hascomplied with at least some of the plurality of surgical guidelines haschanged over time (e.g., whether they are improving in conformance tothe guidelines, etc.). In a further example, the one or more othermetrics can include a likely reason for the failure to comply with aparticular surgical guideline during a particular surgical procedure. Inanother example, the one or more other metrics can be a failure typerelated to a failure to comply with a particular surgical guidelineduring a particular surgical procedure. In some examples, a machinelearning model may be trained using training examples to determineadjustment to the insurance premium based on a level of complianceand/or one or more other metrics. An example of such training examplemay include a sample level of compliance and/or sample additional data,together with a label indicative of a sample adjustment to a sampleinsurance premium. The trained machine learning model may be used toanalyze the determined level of compliance and/or the one or more othermetrics to determine the adjustment to the insurance premium.

In aspects involving a plurality of surgeons and an aggregatedcompliance level, the aggregated compliance level can be output toenable the determination of the adjustment to the insurance premium.

In aspects involving a reference compliance level, the determination ofthe adjustment to the insurance premium can be based on a differencebetween the aggregated compliance level and the reference compliancelevel.

In some aspects, the level of compliance may only be output when thedifference between the determined level of compliance and a historiclevel of compliance is above a selected threshold value.

In some examples, 1110 may include determining, based on the imageanalysis in 1108, that the surgeon complied with a particular surgicalguideline of the plurality of surgical guidelines in a first subset ofthe plurality of intracorporeal video streams, and that the surgeonfailed to comply with the particular surgical guideline in a secondsubset of the plurality of intracorporeal video streams. Further, atemporal relation between the first and second subsets of the pluralityof intracorporeal video streams may be determined. Some non-limitingexamples of such temporal relation may include all the intracorporealvideo streams of the first subset precede all the intracorporeal videostreams of the second subset, at least one of the intracorporeal videostreams of the first subset precedes all the intracorporeal videostreams of the second subset, all the intracorporeal video streams ofthe first subset precede at least one of the intracorporeal videostreams of the second subset, at least one of the intracorporeal videostreams of the first subset precedes at least one of the intracorporealvideo streams of the second subset, all the intracorporeal video streamsof the second subset precede all the intracorporeal video streams of thefirst subset, at least one of the intracorporeal video streams of thesecond subset precedes all the intracorporeal video streams of the firstsubset, all the intracorporeal video streams of the second subsetprecede at least one of the intracorporeal video streams of the firstsubset, at least one of the intracorporeal video streams of the secondsubset precede at least one of the intracorporeal video streams of thefirst subset, and so forth. Further, the adjustment to the insurancepremium of the original insurance policy may be determined based on thetemporal relation between the first and second subsets of the pluralityof intracorporeal video streams. For example, when the temporal relationis a first relation, the adjustment may be a first adjustment, and whenthe temporal relation is a second relation, the adjustment may be asecond adjustment. The second adjustment may differ from the firstadjustment. In another example, when all the intracorporeal videostreams of the second subset precede all the intracorporeal videostreams of the first subset, the adjustment may be a first adjustment,and when all the intracorporeal video streams of the first subsetprecede all the intracorporeal video streams of the second subset, theadjustment may be a second adjustment. The second adjustment may differfrom the first adjustment.

Medical malpractice occurs when a medical or health care professional,through a negligent act or omission, deviates from standards in theirprofession, thereby causing injury or death to a patient. The negligencemight arise from errors in diagnosis, treatment, aftercare or healthmanagement. Surgeons are usually required to carry insurance to protectsurgeons from bearing the full cost of defending against a negligenceclaim made by a patient, and damages awarded in such a civil lawsuit.When a patient alleges medical malpractice, an insurance provider mustassess the patient's allegations to determine whether they havesufficient merit to warrant a settlement or other compensation.

Embodiments expedite processing of medical malpractice claims. In anembodiment, a malpractice claim may be analyzed to identify, in a videocorresponding to the claim, a video portion relevant to resolving theclaim. In some aspects, a non-transitory computer readable medium cancontain instructions that, when executed by at least one processor,cause the at least one processor to execute operations to performintracorporeal video analysis using a medical malpractice claim.

FIG. 12 is a flowchart of an example process 1200 for correlating amedical malpractice claim with a portion of a video, according to someaspects of the present disclosure. It is to be appreciated that not allsteps can be needed to perform the disclosure provided herein. Further,some of the steps can be performed simultaneously, or in a differentorder than shown in FIG. 12 , as will be understood by a person ofordinary skill in the art.

Process 1200 can be implemented by devices, systems, and operationsdescribed in FIGS. 1-11 and using operations caused by computer system2000. Process 1200 can also be understood with reference to FIGS. 13-14. However, process 1200 is not limited to these example aspects.

In 1202, a medical malpractice claim alleging damage from a particularsurgical procedure is received. For example, the medical malpracticeclaim may be read from memory, may be received from an externalcomputing device (e.g., using a digital communication device), may bereceived from an individual (e.g., through a user interface), may beobtained by analyzing an image of at least one page with an OpticalCharacter Recognition algorithm, may be obtained from a database, and soforth.

FIG. 13 a flowchart 1300 depicting an example medical malpractice claim1302. As depicted, medical malpractice claim 1302 alleges that a surgeonperformed gall bladder removal surgery negligently.

Returning to FIG. 12 , in 1204, a linguistic analysis is performed onthe medical malpractice claim. The linguistic analysis is used toidentify a surgical event giving rise to the medical malpractice claim.In one example, the linguistic analysis may be based on operativelanguage used in the medical malpractice claim. In one example, thelinguistic analysis may be conducted using a natural language processingsuch as bag of words, probabilistic context-free grammar, or a LargeLanguage Model. In one example, a machine learning model may be trainedusing training examples to identify surgical events giving rise tomedical malpractice claims based on textual content of the medicalmalpractice claims. An example of such training example may include asample textual content of a sample medical malpractice claim, togetherwith a label indicative of a sample surgical event giving rise to thesample medical malpractice claim. The trained machine learning model maybe used to analyze the medical malpractice claim and identify thesurgical event giving rise to the medical malpractice claim.

In some aspects, the surgical event can be a failure to perform arequired intracorporeal action or can be a manner of performing arequired intracorporeal action.

Flowchart 1300 of FIG. 13 shows an example linguistic analysis 1304performed on medical malpractice claim 1302. This linguistic analysiscan be performed as described for 1204, and can be used to identify asurgical event 1306. For example, when linguistic analysis 1304 isperformed on medical malpractice claim 1302, it is identified that abile duct was damaged. Further, damaging a bile duct is known to beassociated with clipping, cutting and/or transecting a ductal structure.Therefore, clipping, cutting and/or transecting a ductal structure isidentified as surgical event 1306. Surgical event 1306 can then be usedthroughout subsequent operations.

Returning to FIG. 12 , in 1206, an intracorporeal video stream depictingthe particular surgical procedure is accessed. For example, theintracorporeal video stream may be read from memory, may be receivedfrom an external computing device (e.g., using a digital communicationdevice), may be captured using an image sensor, may be obtain from arepository (for example, as described above), and so forth.

FIG. 14 is an illustration 1400 of intracorporeal video streams,according to some aspects of the present disclosure. Illustration 1400includes an intracorporeal video stream 1402.

Returning to FIG. 12 , in 1208, based on the identified surgical eventin 1204, the intracorporeal video stream is analyzed to identify aseries of frames from the intracorporeal video stream that depicts thesurgical event. In another example, textual content of the medicalmalpractice claim and the intracorporeal video stream may be analyzed(for example, using a multimodal model) to identify a series of framesfrom the intracorporeal video stream that depicts a surgical eventassociated with the medical malpractice claim.

In some aspects, a machine learning model may be trained using trainingexamples to identify the series of frames corresponding to surgicalevents in videos based on surgical events and/or operative languageand/or other information. An example of such training example mayinclude a sample surgical event and/or a sample operative languageand/or a sample other information corresponding to the sample surgicalevent, and a sample surgical video, together with a label indicatingframes of the sample surgical video that corresponds to the samplesurgical event. In one example, the trained machine learning model maybe used to analyze the intracorporeal video stream based on theidentified surgical event and/or the identified operative languageand/or the other information to identify the series of frames from theintracorporeal video stream that depicts the surgical event. In anotherexample, the trained machine learning model may be used to analyzetextual content of the medical malpractice claim and the intracorporealvideo to identify the series of frames from the intracorporeal videostream that depicts a surgical event associated with the medicalmalpractice claim.

In some examples, a convolution of at least part of the intracorporealvideo stream may be calculated to obtain a result value. Further, basedon the identified surgical event and the result value, it may bedetermined whether to include a particular frame of the intracorporealvideo stream in the series of frames from the intracorporeal videostream that depicts the surgical event. In one example, the at leastpart of the intracorporeal video stream may be at least part of theparticular frame of the intracorporeal video stream. In one example,when the identified surgical event is a particular event and the resultvalue is a first numerical value, the particular frame may be includedin the series of frames, and when the identified surgical event is theparticular event and the result value is a second numerical value, theparticular frame may be excluded from the series of frames. In anotherexample, the textual content of the medical malpractice claim may beanalyzed to determine a mathematical object, for example using an NLPalgorithm or a text embedding algorithm. Further, based on themathematical object and the result value, it may be determined whetherto include a particular frame of the intracorporeal video stream in theseries of frames from the intracorporeal video stream that depicts thesurgical event. For example, when the mathematical object is aparticular numerical value and the result value is a first numericalvalue, the particular frame may be included in the series of frames, andwhen the numerical value is the particular numerical value and theresult value is a second numerical value, the particular frame may beexcluded from the series of frames. In another example, when themathematical object is a first numerical value and the result value is aparticular numerical value, the particular frame may be included in theseries of frames, and when the numerical value is a second numericalvalue and the result value is the particular numerical value, theparticular frame may be excluded from the series of frames. In yetanother example, when the mathematical object is a first numerical valueand the result value is a second numerical value, the particular framemay be included in the series of frames, and when the numerical value isa third numerical value and the result value is a fourth numericalvalue, the particular frame may be excluded from the series of frames.

In some aspects, a video search query is determined based on thereimbursement code and/or the identified surgical event. Accordingly,the analysis in 1208 uses the video search query to search in theintracorporeal video stream for the series of frames corresponding tothe surgical event. For example, the video search query may include ashort video depicting the identified surgical event in a differentsurgical procedure, and searching using the video search query mayinclude searching the intracorporeal video stream for frames similar tothe frames of the short video. In another example, the video searchquery may include an indication of one or more elements of theidentified surgical event, such as medical equipment or anatomicalstructure, and searching using the video search query may includesearching the intracorporeal video stream for frames depicting the oneor more elements. In yet another example, the video search query mayinclude an indication of a portion of the intracorporeal video stream(e.g., the portion between two other events depicted in theintracorporeal video stream), and searching using the video search querymay include searching only for frames in the portion of theintracorporeal video stream. In one example, a data-structureassociating reimbursement codes and/or surgical events with video searchqueries may be accessed based on the reimbursement code and/or theidentified surgical event to obtain the video search query. In anotherexample, a machine learning model may be trained using training examplesto generate video search queries based on reimbursement codes and/orsurgical events. An example of such training example may include asample reimbursement code and/or a sample surgical event, together witha sample video search query. The trained machine learning model may beused to analyze the reimbursement code and/or the identified surgicalevent to generate the video search query.

In some aspects, a second series of frames depicting a prerequisite tothe surgical event can be output and used in subsequent operations. Forexample, when the surgical event is clipping and/or cutting ananatomical structure, the prerequisite may include reaching CVS beforethe clipping and/or cutting the anatomical structure. In one example,the prerequisite may be determined based on the surgical event. Forexample, a data-structure associating surgical events with prerequisitesmay be accessed based on the identified surgical event to determine theprerequisite. In one example, the prerequisite may be determined basedon the surgical event and the medical malpractice claim. For example, adata-structure associating surgical events and/or medical malpracticeclaims with prerequisite may be accessed based on the surgical eventand/or the medical malpractice claim to determine the prerequisite. Inone example, the medical malpractice claim may indicate a damage to abile duct, the surgical event may be clipping and/or cutting ananatomical structure, and the prerequisite may be a prerequisite knownto reduce the risk of damage to the bile duct in clipping and/or cuttingan anatomical structure, such as reaching CVS. In one example, thesecond series of frames depicting the determined prerequisite may beidentified by analyzing the intracorporeal video stream using a machinelearning model. The machine learning model may be a machine learningmodel trained using training examples. An example of such trainingexample may include a sample prerequisite and a sample surgical video,together with a label indicating a series of frames from the samplesurgical video depicting the sample prerequisite.

With reference to illustration 1400 of FIG. 14 , a series of frames 1404from intracorporeal video stream 1402 can be identified based on, forexample, surgical event 1306 of FIG. 3 . For example, it can bedetermined that series of frames 1404 depicts bile duct damage.

Returning to FIG. 12 , in 1210, based on the identified series of framesin 1208, an action corresponding to the medical malpractice claim isinitiated.

In some aspects, the action includes providing an indication whether acorrelation exists between the series of frames and the medicalmalpractice claim. In one example, the series of frames may be analyzedto determine whether a correlation exists between the series of framesand the medical malpractice claim. For example, a machine learning modelmay be trained using training examples to determine whether correlationsexist between surgical footage and medical malpractice claims. Anexample of such training example may include a sample surgical footageand a sample medical malpractice claim, together with a label indicativeof whether a correlation exists between the sample surgical footage andthe sample medical malpractice claim. The trained machine learning modelmay be used to analyze the series of frames and the medical malpracticeclaim to determine whether a correlation exists between the series offrames and the medical malpractice claim.

In some aspects, the action includes automatically determining whether abasis exists for the medical malpractice claim. In one example, theseries of frames may be analyzed to determine whether a basis exists forthe medical malpractice claim. For example, a machine learning model maybe trained using training examples to determine whether a basis existsfor medical malpractices claims from surgical footage. An example ofsuch training example may include a sample surgical footage and a samplemedical malpractice claim, together with a label indicative of whether abasis exists for the sample medical malpractice claim. The trainedmachine learning model may be used to analyze the series of frames andthe medical malpractice claim to determine whether a basis exists forthe medical malpractice claim.

In other aspects, the action includes transmitting the series of framesto an entity for consideration. For example, the series of frames may betransmitted to an external computing device associated with the entityusing a digital communication device. In some aspects, the action mayinclude presenting the series of frames to an individual forconsideration. For example, the series of frames may be presented via auser interface, using a display screen, using an extended realityappliance, and so forth.

When the action is initiated, a recommendation related to the medicalmalpractice claim can be determined and the recommendation can beoutput. In one example, the recommendation may be determined based on ananalysis of the medical malpractice claim and/or on an analysis theintracorporeal video stream depicting the particular surgical procedureand/or on an analysis of the identified series of frames from theintracorporeal video stream. In some examples, a machine learning modelmay be trained using training examples to generate recommendationrelated to medical malpractice claims based on the medical malpracticeclaims and/or surgical footage. An example of such training example mayinclude a sample medical malpractice claim and a sample surgicalfootage, together with a sample recommendation related to the samplemedical malpractice claim. The trained machine learning model may beused to analyze the medical malpractice claim and/or the intracorporealvideo stream depicting the particular surgical procedure and/or theidentified series of frames from the intracorporeal video stream togenerate the recommendation. In some examples, a convolution of at leastpart of the intracorporeal video stream or at least part of the seriesof frames may be calculated to obtain a result value. Further, therecommendation may be generated based on the result value. For example,a function of the result value may be used as an input to a generativemodel (such as an LLM) to generate the recommendation.

The recommendation can be determined by accessing a textual medicalrecord related to the medical malpractice claim, where therecommendation is based on the textual medical record. The textualmedical record can be, for example, at least one of a postoperativereport of the particular surgical procedure or an electronic medicalrecord of a patient on who the particular surgical procedure wasperformed. For example, the textual medical record may be analyzed todetermine a mathematical object, for example using text embeddingalgorithm, and the mathematical object may be used to generate therecommendation. For example, a function of the mathematical object maybe used as an input to a generative model (such as an LLM) to generatethe recommendation. Determining the recommendation can also be based ona time elapsed since the particular surgical procedure, a patienttreated by the particular surgical procedure (e.g., a patient's age,medical condition, etc.), or a surgeon performing the particularsurgical procedure. In some examples, the recommendation may be based oninformation related to the medical malpractice claim, such as anycombination of an analysis of the medical malpractice claim and/or ananalysis of the textual medical record related to the medicalmalpractice claim and/or an analysis of the intracorporeal video streamdepicting the particular surgical procedure and/or the time elapsedsince the particular surgical procedure and/or the patient treated bythe particular surgical procedure and/or the surgeon performing theparticular surgical procedure and/or whether the surgeon is currentlyaffiliated with the medical facility where the particular surgicalprocedure was performed. For example, a machine learning model may betrained using training examples to generate recommendation related tomedical malpractice claims based on such information associated with themedical malpractice claims. An example of such training example mayinclude a sample information associated with a sample medicalmalpractice claim, together with a sample recommendation related to thesample medical malpractice claim. The trained machine learning model maybe used to generate the recommendation based on any combination of themedical malpractice claim and/or the textual medical record related tothe medical malpractice claim and/or the intracorporeal video streamdepicting the particular surgical procedure and/or the time elapsedsince the particular surgical procedure and/or the patient treated bythe particular surgical procedure and/or the surgeon performing theparticular surgical procedure and/or whether the surgeon is currentlyaffiliated with the medical facility where the particular surgicalprocedure was performed. For example, a machine learning model may betrained using training examples to generate recommendation related tomedical malpractice claims based on such information associated with themedical malpractice claims.

In some aspects, determining the recommendations can include analyzingthe intracorporeal video stream and the textual medical record based onthe medical malpractice claim to determine a likelihood that acomplication alleged in the medical malpractice claim is a result of anerror made during the particular surgical procedure. In these aspects,the recommendation can be based on the determined likelihood. Forexample, a multimodal regression model may be used to analyze theintracorporeal video stream and/or the textual medical record and/or themedical malpractice claim to determine the likelihood that thecomplication alleged in the medical malpractice claim is the result ofthe error made during the particular surgical procedure. In one example,the multimodal regression model may be a machine learning model trainedusing training examples. An example of such training example may includea sample intracorporeal video stream depicting a sample surgicalprocedure and/or a sample textual medical record associated with thesample surgical procedure and/or a sample medical malpractice claimassociated with the sample surgical procedure to determine thelikelihood that a sample complication alleged in the sample medicalmalpractice claim is a result of an error made during the samplesurgical procedure.

The particular surgical procedure can be performed at a medicalfacility, and the recommendation can be determined based on the medicalfacility the surgeon was affiliated with at the time of the particularsurgical procedure, for example using a machine learning model asdescribed above. In these aspects, determining the recommendation canalso be based on whether the surgeon is currently affiliated with themedical facility.

In some examples, the recommendation can be a recommendation related toa decision whether or not to settle the medical malpractice claim. Forexample, a machine learning model may be trained using training examplesto determine whether to recommend to settle medical malpractice claimsbased on information associated with the medical malpractice claims.Such information associated with the medical malpractice claims aredescribed above. An example of such training example may includeinformation associated with a sample medical malpractice claim, togetherwith a sample recommendation on whether or not to settle the samplemedical malpractice claim.

In some examples, the recommendation may include an indication ofexpected litigation costs associated with a prospective litigation ofthe medical malpractice claim. For example, a multimodal regressionmodel may be used to analyze information associated with the medicalmalpractice claim to determine the expected litigation costs associatedwith the prospective litigation of the medical malpractice claim. Suchinformation associated with the medical malpractice claims are describedabove. In one example, the multimodal regression model may be a machinelearning model trained using training examples. An example of suchtraining example may include sample information associated with a samplemedical malpractice claim, together with a label indicating expectedlitigation costs associated with prospective litigation of the samplemedical malpractice claim.

In some examples, the recommendation may include an indication of apredicted likelihood of success associated with a prospective litigationof the medical malpractice claim. For example, a multimodal regressionmodel may be used to analyze information associated with the medicalmalpractice claim to determine the predicted likelihood of successassociated with the prospective litigation of the medical malpracticeclaim. Such information associated with the medical malpractice claimsare described above. In one example, the multimodal regression model maybe a machine learning model trained using training examples. An exampleof such training example may include sample information associated witha sample medical malpractice claim, together with a label indicating apredicted likelihood of success associated with prospective litigationof the sample medical malpractice claim.

In some examples, the recommendation may include an indication apredicted compensation associated with a prospective lawsuit associatedwith the medical malpractice claim. For example, a multimodal regressionmodel may be used to analyze information associated with the medicalmalpractice claim to determine the predicted compensation associatedwith the successful lawsuit associated with the medical malpracticeclaim. Such information associated with the medical malpractice claimsare described above. In one example, the multimodal regression model maybe a machine learning model trained using training examples. An exampleof such training example may include sample information associated witha sample medical malpractice claim, together with a label indicating apredicted compensation associated with a successful lawsuit associatedwith the sample medical malpractice claim.

In this way, video can be quickly determined to speed processing ofmedical malpractice claims.

Health insurance or medical insurance is a type of insurance that coversthe whole or a part of the risk of a person incurring medical expenses.Medical expenses are often described in payment and billing contextsusing standard claim codes. Some non-limiting examples of claim codesets may include the Current Procedural Terminology (CPT), Level IIHCPCS, ICD9, ICD10, and so forth.

To obtain full reimbursement for all medical procedures, surgeons orother medical entities may be required to include codes in their billsfor each surgical event in a procedure for which insurance reimbursementis sought. When questions arise about whether the codes are accurate,proof may be required. Intracorporeal video footage may be analyzed todetect reimbursable events, and image data demonstrating that thereimbursable surgical event actually occurred may be output. In someaspects, a non-transitory computer readable medium can containinstructions that, when executed by at least one processor, cause the atleast one processor to execute operations to perform intracorporealvideo analysis for identifying image data depicting a reimbursableevent.

In addition to using video to expedite processing of medical malpracticeclaims, embodiments may expedite processing of medical insurance claims.To expedite processing of medical insurance claims, a medical claim codemay be analyzed to identify, in a video corresponding to the claim, avideo portion relevant to resolving the claim. In some aspects, anon-transitory computer readable medium can contain instructions that,when executed by at least one processor, cause the at least oneprocessor to execute operations to perform intracorporeal video analysisusing a medical insurance claim.

FIG. 15 is a flowchart of an example process for correlating a medicalclaim code with a portion of a video, according to some aspects of thepresent disclosure. It is to be appreciated that not all steps can beneeded to perform the disclosure provided herein. Further, some of thesteps can be performed simultaneously, or in a different order thanshown in FIG. 15 , as will be understood by a person of ordinary skillin the art.

Process 1500 can be implemented by devices, systems, and operationsdescribed in FIGS. 1-14 and using operations caused by computer system2000. Process 1500 can also be understood with reference to FIG. 16 .However, process 1500 is not limited to these example aspects.

In 1502, a medical insurance claim is received. The medical insuranceclaim arises from a particular surgical procedure. For example, themedical insurance claim may be read from memory, may be received from anexternal computing device (e.g., using a digital communication device),may be received from an individual (e.g., through a user interface), maybe obtained by analyzing an image of at least one page with an OpticalCharacter Recognition algorithm, may be obtained from a database, and soforth.

In 1504, a reimbursement code from the medical insurance claim isaccessed. A reimbursement code can be a standardized code determined forevery respective feature of every respective surgical procedure.Additional, a reimbursement code can be a standardized code determinedfor every respective piece of equipment used in every respectivesurgical procedure.

FIG. 16 is an illustration of an example process 1600 for correlating amedical claim code 1604 with a portion of a video 1606, according tosome aspects of the present disclosure. As shown in process 1600, areimbursement code can be, for example, reimbursement code 1604.Reimbursement code 1604 can correspond to a medical insurance claim. Inthis example, reimbursement code 1604 is directed towards laparoscopicsurgery and cholecystectomy with cholangiography.

Returning to FIG. 15 , in 1506, an intracorporeal video stream depictingthe particular surgical procedure is accessed. For example, theintracorporeal video stream may be read from memory, may be receivedfrom an external computing device (e.g., using a digital communicationdevice), may be captured using an image sensor, may be accessed in arepository as described above, and so forth. In one example, at the timeof accessing the intracorporeal video stream depicting the particularsurgical procedure, the particular surgical procedure may be completedor in progress.

Process 1600 illustrates an example intracorporeal video stream asintracorporeal video stream 1602.

Returning to FIG. 15 , in 1508, based on the reimbursement code from1504, the intracorporeal video stream is analyzed to identify a seriesof frames from the intracorporeal video stream. The series of framesdepicts a surgical event related to the reimbursement code. In someexamples, a machine learning model may be trained using trainingexamples to identify frames depicting events in videos based onreimbursement codes. An example of such training example may include asample surgical video and a sample reimbursement code, together with alabel indicating a sample selection of frames depicting eventsassociated with the sample reimbursement code in the sample surgicalvideo. The trained machine learning model may be used to analyze thereimbursement code from 1504 and the intracorporeal video stream toidentify the series of frames from the intracorporeal video stream. Insome examples, a data-structure associating reimbursement codes withsurgical events may be accessed based on the reimbursement code from1504 to determine a particular surgical event. Further, theintracorporeal video stream may be analyzed to identify particularframes from the intracorporeal video stream depicting the particularsurgical event, and the particular frames may be included in theidentified series of frames. In some examples, a convolution of at leastpart of a particular frame of the intracorporeal video stream may becalculated to obtain a result value. Further, based on the result value,it may be determined whether to include the particular frame in theidentified series of frames. In one example, when the result value is afirst numerical value, the particular frame may be included in theidentified series of frames, and when the result value is a secondnumerical value, the particular frame may be excluded from theidentified series of frames. In another example, a threshold may beselected based on the reimbursement code from 1504, the particular framemay be included in the identified series of frames when the result valueis above the selected threshold, and the particular frame may beexcluded from the identified series of frames when the result value isbelow the selected threshold.

In some aspects, a video search query is determined based on thereimbursement code. Accordingly, the analysis in 1508 uses the videosearch query to search in the intracorporeal video stream for the seriesof frames corresponding to the surgical event. For example, the videosearch query may include a short video depicting the surgical event in adifferent surgical procedure, and searching using the video search querymay include searching the intracorporeal video stream for frames similarto the frames of the short video. In another example, the video searchquery may include an indication of one or more elements of the surgicalevent, such as medical equipment or anatomical structure, and searchingusing the video search query may include searching the intracorporealvideo stream for frames depicting the one or more elements. In yetanother example, the video search query may include an indication of aportion of the intracorporeal video stream (e.g., the portion betweentwo other events depicted in the intracorporeal video stream), andsearching using the video search query may include searching only forframes in the portion of the intracorporeal video stream.

In some aspects, a second series of frames depicting a prerequisite tothe surgical event can be output and used in subsequent operations. Inone example, the prerequisite may be determined based on the surgicalevent and/or the medical insurance claim and/or the reimbursement code.For example, a data-structure associating surgical events and/or medicalinsurance claims and/or reimbursement codes with prerequisites may beaccessed based on the surgical event and/or the medical insurance claimand/or the reimbursement code to determine the prerequisite. In oneexample, the reimbursement code may indicate a laparoscopiccholecystectomy, the surgical event may be clipping and/or cutting ananatomical structure, and the prerequisite may be a prerequisite knownto reduce the risk of damage to the bile duct in clipping and/or cuttingan anatomical structure, such as reaching CVS. In one example, thesecond series of frames depicting the determined prerequisite may beidentified by analyzing the intracorporeal video stream using a machinelearning model. The machine learning model may be a machine learningmodel trained using training examples. An example of such trainingexample may include a sample prerequisite and a sample surgical video,together with a label indicating a series of frames from the samplesurgical video depicting the sample prerequisite.

As shown in process 1600, intracorporeal video stream 1602 can beanalyzed based on reimbursement code 1604, directed towards laparoscopyand cholecystectomy. Accordingly, a series of frames 1606 can beidentified from intracorporeal video stream 1602. The series of frames1606 is only a portion of video stream 1602, and depict the surgicalevent of a laparoscopic cholecystectomy with cholangiography.

Returning to FIG. 15 , in 1510, based on the identified series of framesfrom 1508, an action is initiated that corresponds to the medicalinsurance claim from 1502.

In some aspects, the action can include providing an indication ofwhether a reimbursable correlation exists between the series of framesand the reimbursement code. For example, a machine learning model may betrained using training examples to determine whether reimbursablecorrelation exists between surgical footage and reimbursement codes. Anexample of such training example may include a sample series of framesand a sample reimbursement code, together with a label indicatingwhether there is a reimbursable correlation between the sample series offrames and the sample reimbursement code. Further, the machine learningmodel may be used to analyze the series of frames and the reimbursementcode to determine whether a reimbursable correlation exists between theseries of frames and the reimbursement code.

In some aspects, the action can include determining whether a basisexists for the medical insurance claim. For example, a machine learningmodel may be trained using training examples to determine whether abasis exists for medical insurance claims in surgical footage. Anexample of such training example may include a sample medical insuranceclaim and a sample series of frames, together with a label indicatingwhether a basis exists for the sample medical insurance claim in thesample series of frames. The trained machine learning model may be usedto analyze the medical insurance claim and the series of frames todetermine whether a basis exists for the medical insurance claim.

In some aspects, the action can include transmitting (for example, usinga digital communication device) the series of frames from 1508 to anentity for consideration. Some non-limiting examples of such entity mayinclude an insurer, a medical coding auditor, a medical coder, anaccounting department, and so forth. In other examples, the action mayinclude presenting the series of frames from 1508 to an individual forconsideration. For example, the series of frames may be presented via auser interface, using a display screen, using an extended realityappliance, and so forth.

When the action is initiated, a recommendation related to the medicalinsurance claim can be determined and the recommendation can be output.In one example, the recommendation may include a recommendation toaccept the medical insurance claim. In another example, therecommendation may include a recommendation to reject the medicalinsurance claim. In yet another example, the recommendation may includea recommendation to conduct additional investigation related to themedical insurance claim. In yet another example, the recommendation mayinclude a recommendation to modify the reimbursement code in the medicalinsurance claim and/or to add another reimbursement code to the medicalinsurance claim and/or to remove the reimbursement code from the medicalinsurance claim.

In some aspects, the recommendation can be determined by accessing atextual medical record of the medical insurance claim, where therecommendation is based on the textual medical record. The textualmedical record can be, for example, a postoperative report of theparticular surgical procedure or an electronic medical record of apatient on who the particular surgical procedure was performed. Forexample, the textual medical record may be analyzed using a LargeLanguage Model to generate the recommendation. In another example, amultimodal model may be used to analyze the textual medical record andthe intracorporeal video stream to generate the recommendation.

In some aspects, the recommendation can be based on an analysis of theintracorporeal video stream. For example, a convolution of at least partof the intracorporeal video stream can be calculated to obtain a resultvalue, where the result value is used to determine the recommendation.In another example, a machine learning model can be used to analyze theintracorporeal video stream to determine the recommendation. The machinelearning model can be trained similarly to the model described withreference to FIG. 12 . In another example, an LLM may be used to analyzethe medical insurance claim and the textual medical record to generatethe recommendation.

In some aspects, the recommendation can include a recommendation to addan additional reimbursement code to the medical insurance claim. Then,the intracorporeal video stream can be analyzed to select the additionalreimbursement code. In other aspects, the recommendation can include arecommendation to substitute the reimbursement code with an alternativereimbursement code to the medical insurance claim. Then, theintracorporeal video stream can be analyzed to select the alternativereimbursement code.

In some aspects, the reimbursement code can include a firstreimbursement code and a second reimbursement code. If a recommendationis determined, then the recommendation can include a firstrecommendation related to the first reimbursement code and a secondrecommendation related to the second reimbursement code. In theseaspects, the second recommendation differs from the firstrecommendation. For example, the first recommendation can be arecommendation to remove the first reimbursement code from the medicalinsurance claim. In another example, the first recommendation can be arecommendation to deny reimbursements from the first reimbursement code.In this example, the second recommendation can be a recommendation toapprove reimbursements from the second reimbursement code.

As mentioned above, to obtain full reimbursement for all medicalprocedures, surgeons may be required to include codes in their bills foreach surgical event in a procedure for which insurance reimbursement issought. Identifying which codes are needed can be tedious and errorprone. Embodiments provide computer readable mediums, methods andsystems to automatically determine insurance codes from video byidentifying the tools used in the video.

FIG. 17 is a flowchart of an example process for analyzing surgicalvideo to support insurance reimbursement, according to some aspects ofthe present disclosure. It is to be appreciated that not all steps canbe needed to perform the disclosure provided herein. Further, some ofthe steps can be performed simultaneously, or in a different order thanshown in FIG. 17 , as will be understood by a person of ordinary skillin the art.

Process 1700 can be implemented by devices, systems, and operationsdescribed in FIGS. 1-16 and using operations caused by computer system2000. Process 1700 can also be understood with reference to FIG. 18 .However, process 1700 is not limited to these example aspects.

In 1702, intracorporeal video footage captured during a surgicalprocedure on a patient is accessed. For example, the intracorporealvideo footage may be read from memory, may be received from an externalcomputing device (e.g., using a digital communication device), may becaptured using at least one image sensor, may be accessed in arepository as described above, and so forth. In one example, at the timeof accessing the intracorporeal video footage, the particular surgicalprocedure may be completed or in progress.

FIG. 18 is an illustration of an example process 1800 for analyzingsurgical video to support insurance reimbursement, according to someaspects of the present disclosure. As shown, process 1800 includesintracorporeal video stream 1802. Intracorporeal video stream 1802 canbe the video footage accessed in 1702.

In 1704, the intracorporeal video footage is analyzed to detect asurgical tool. Computer vision may be used to identify one or moremedical instruments used in a surgical procedure. Objectdetection/recognition is a computer vision technique that allows us toidentify and locate objects in an image or video. With this kind ofidentification and localization, object detection can be used to countobjects in a scene and determine and track their precise locations, allwhile accurately labeling them. An example object detection algorithm isthe Viola-Jones algorithm. In one example, a convolution of at leastpart of the intracorporeal video footage may be calculated to obtain aresult value. Further, the detection of the surgical tool may be basedon the result value. For example, when the result value is a firstnumerical value, the surgical tool may be detected, and when the resultvalue is a second numerical value, no surgical tool may be detected. Inanother example, when the result value is a first numerical value, asurgical tool of a first type may be detected, and when the result valueis a second numerical value, a surgical tool of a second type may bedetected. The second type may differ from the first type.

Process 1800 depicts the analysis of intracorporeal video stream 1802.For example, intracorporeal video stream 1802 can be analyzed forvarious surgical tools, such as a clip applier (i.e., tool 1806A),scissors (i.e., tool 1806B), or a dissector (i.e., tool 1806C).

In 1706, the intracorporeal video footage is analyzed to detect areimbursable event associated with the surgical tool from 1704 withinthe intracorporeal video footage. In one example, a convolution of atleast part of the intracorporeal video footage may be calculated toobtain a result value. Further, the detection of the reimbursable eventmay be based on the result value. For example, when the result value isa first result value, the reimbursable event may be detected, and whenthe result value is a second result value, no reimbursable event may bedetected. In another example, when the result value is a first resultvalue, a first reimbursable event may be detected, and when the resultvalue is a second result value, a second reimbursable event may bedetected. The second reimbursable event may differ from the firstreimbursable event.

Frames of the intracorporeal video footage may be analyzed to detect areimbursable event, such as a reimbursable event associated with asurgical tool. For example, a machine learning model may be trainedusing training examples to detect reimbursable events (such asreimbursable events associated with surgical tools) in images and/orvideos. An example of such training example may include a sampleintracorporeal image or video depicting a sample surgical tool, togetherwith a label indicating a reimbursable event associated with the samplesurgical tool in the sample intracorporeal image or video, a labelindicating a frame or a portion of the sample intracorporeal image orvideo representative of the reimbursable event in the sampleintracorporeal image or video, and/or a label indicating a medicalreimbursement code corresponding to the reimbursable event in the sampleintracorporeal image or video. The trained machine learning model may beused to analyze the intracorporeal video footage and detect thereimbursable event within the intracorporeal video footage. At least oneframe of the intracorporeal video footage representative of the detectedreimbursable event may be identified, for example using the same machinelearning model. In some examples, the at least one representative framemay be extracted from the intracorporeal video footage.

Based on identification of the medical instrument, a particularreimbursable event may be identified at a location in the video footagecorresponding to the medical instrument. Here, a location may refer toone or more frames in which the medical instrument appears and/or toportions of the frames depicting the medical instrument or surroundingthe depiction of the medical instrument. For example, a scalpel or otherinstrument may indicate that an incision is being made and a markeridentifying the incision may be included in the timeline at thislocation. In some aspects, anatomical structures may be identified inthe video footage using the computer analysis. For example, thedisclosed methods may include identifying organs, tissues, fluids orother structures of the patient to determine a reimbursable event. Insome aspects, a reimbursable event may be determined based on aninteraction between a medical instrument and the anatomical structure.For example, visual action recognition algorithms may be used to analyzethe video and detect the interactions between the medical instrument andthe anatomical structure. Other examples of features that may be used todetermine a reimbursable event may include, motions of a surgeon orother medical professional, patient characteristics, surgeoncharacteristics or characteristics of other medical professionals,sequences of operations being performed, timings of operations orevents, characteristics of anatomical structures, or medical conditions.

The detected reimbursable event can be or be based on at least one of atype of medical procedure that took place during the surgical procedure,or a complication that took place during the surgical procedure. In oneexample, the type of medical procedure that took place during thesurgical procedure may be determined by analyzing the intracorporealvideo footage using a visual classification algorithm to one of aplurality classes, where each class corresponds to a type of medicalprocedures, thereby determining the type of medical procedure that tookplace during the surgical procedure. In one example, the complicationmay be detected by analyzing the intracorporeal video footage. Forexample, a machine learning model may be trained using training examplesto detect complications in surgical footage. An example of such trainingexample may include a sample surgical footage, together with a labelindicating whether the sample surgical footage depicts a complicationand/or the type of the complication. The trained machine learning modelmay be used to analyze the intracorporeal video footage to detect thecomplication.

In some examples, the detected reimbursable event can be or include asurgical action performed in the surgical procedure, using a particularsurgical tool (such as the surgical tool from 1704). In one example, thesurgical action may be detected by analyzing the intracorporeal videofootage using a visual action recognition algorithm. In some aspects, asurgical action can be at least one of removing material, stitchingmaterial, stopping blood flow, restarting blood flow, stitching, etc.

In some examples, the detected reimbursable event can include aninteraction between the surgical tool and an anatomical structure of thepatient. Some non-limiting examples of such interaction may include aphysical contact between the surgical tool and the anatomical structure,a force applied by or via the surgical tool on the anatomical structure,a manipulation on the anatomical structure performed by or using thesurgical tool, an interaction that is part of a surgical action, and soforth. For example, the interaction between the surgical tool and theanatomical structure may be detected by analyzing the intracorporealvideo footage. For example, a region of a frame of the intracorporealvideo footage depicting the surgical tool may be identified (forexample, using semantic segmentation algorithm), a region of the frameof the intracorporeal video footage depicting the anatomical structuremay be identified (for example, using semantic segmentation algorithm),and the interaction may be detected based on a distance between the tworegions and/or the relative orientation between the two regions.

Process 1800 depicts analysis of intracorporeal video stream 1802. Forexample, intracorporeal video stream 1802 can be analyzed to detectreimbursable events corresponding to surgical tools, such as a clipapplier (i.e., tool 1806A), scissors (i.e., tool 1806B), or a dissector(i.e., tool 1806C). The events can be those depicted in frame 1804A,frame 1804B, and frame 1804C, where frame 1804A depicts use of tool1806A, frame 1804B depicts use of tool 1806B, and frame 1804C depictsuse of tool 1806C.

Returning to FIG. 17 , in 1708, based on the surgical tool from 1704, amedical reimbursement code related to the detected reimbursable eventfrom 1706 is determined. The medical reimbursement code can be auniversal medical alphanumeric code. In some aspects, the medicalreimbursement code can be determined using the same machine learningmodel from 1706. The trained machine learning model may be used toanalyze the intracorporeal video footage and determine the medicalreimbursement code related to the detected reimbursable event. In someaspects, a data-structure associating surgical tools with medicalreimbursement codes may be accessed based on the surgical tool from 1704to determine the medical reimbursement code related to the detectedreimbursable event. In one example, the data-structure may identify aplurality of alternative medical reimbursement codes related to thesurgical tool from 1704. Further, the intracorporeal video footage maybe analyzed to select the medical reimbursement code related to thedetected reimbursable event of the plurality of alternative medicalreimbursement codes. For example, a convolution of at least part of theintracorporeal video footage may be may be calculated to obtain a resultvalue. Further, the selection of the medical reimbursement code relatedto the detected reimbursable event of the plurality of alternativemedical reimbursement codes may be based on the result value. In oneexample, when the result value is a first numerical value, the medicalreimbursement code related to the detected reimbursable event may be afirst medical reimbursement code of the plurality of alternative medicalreimbursement codes, and when the result value is a second numericalvalue, the medical reimbursement code related to the detectedreimbursable event may be a second medical reimbursement code of theplurality of alternative medical reimbursement codes. The second medicalreimbursement code may differ from the first medical reimbursement code.In some examples, the selection of the medical reimbursement coderelated to the detected reimbursable event of the plurality ofalternative medical reimbursement codes may be based on at least one ofa type of medical procedure that took place during the surgicalprocedure, a complication that took place during the surgical procedure,a surgical action performed in the surgical procedure, or an interactionbetween the surgical tool and an anatomical structure of the patient.

In some aspects, the medical reimbursement code can be based on a typeof medical procedure that took place during the surgical procedure, acomplication that took place during the surgical procedure, a surgicalaction performed in the surgical procedure, or an interaction betweenthe surgical tool and an anatomical structure of the patient.

In aspects involving an interaction between the surgical tool and ananatomical structure of the patient, the intracorporeal video footagecan be analyzed to detect the anatomical structure, for example using avisual object detection algorithm. The intracorporeal video footage canalso be analyzed to determine a type of the interaction between thesurgical tool and the anatomical structure. For example, a visualclassification algorithm may be used to analyze the intracorporeal videofootage and classify the interaction to one of a plurality of classes,where each class may correspond to a type of interaction, and therebythe type of the interaction may be determined. Further, this type can beused when determining the medical reimbursement code. For example, whenthe type of the interaction is a first type, the medical reimbursementcode may be determined to be a first code, and when the type of theinteraction is a second type, the medical reimbursement code may bedetermined to be a second code. The second code may differ from thefirst code.

In some aspects, the intracorporeal video footage can be analyzed todetect multiple anatomical structures. For example, a first anatomicalstructure and a second anatomical structure. For example, the anatomicalstructures may be detected by analyzing the intracorporeal video footageusing a visual object detection algorithm. The first anatomicalstructure and the second anatomical structure can be different portionsof a same organ of the patient, or different organs of the patient.Then, the intracorporeal video footage can be analyzed to detectmultiple interactions, for example as described above. For example, afirst interaction can be detected as an interaction between the surgicaltool and the first anatomical structure. A second interaction can alsobe detected as an interaction between the surgical tool and the secondanatomical structure. The determination of the medical reimbursementcode can then be based, at least partially, on whether the firstinteraction precedes the second interaction in the surgical procedure.For example, when the first interaction precedes the second interactionin the surgical procedure, the medical reimbursement code may bedetermined to be a first code, and when the second interaction precedesthe first interaction in the surgical procedure, the medicalreimbursement code may be determined to be a second code. The secondcode may differ from the first code.

In some aspects, the intracorporeal video footage can be analyzed todetect a second surgical tool, for example as described above inrelation to 1704. In one example, the reimbursable event may correspondto a surgical action performed by, or at least involving, both thesurgical tool and the second surgical tool. The determination of themedical reimbursement code can then be based, at least partially, on thesecond surgical tool. For example a data-structure associating pairs ofsurgical tools with medical reimbursement codes may be accessed based onthe surgical tool and the second surgical tool to determine the medicalreimbursement code.

In some aspects, pre-procedure information related to the surgicalprocedure can be accessed. The pre-procedure information can beinformation available before a beginning of the surgical procedure. Forexample, the pre-procedure information may be read from memory, may beaccessed in a database, may be accessed via an external computing device(for example, using a digital communication device), and so forth. Somenon-limiting examples of such pre-procedure information may includemedical records of the patient, surgical planning data, medical imagesof the patient, and so forth. Based on this pre-procedure information,the intracorporeal video footage can be analyzed to identify an elevatedcomplexity of the surgical procedure. An elevated complexity can beelevated compared to a most likely complexity based on the pre-procedureinformation alone. In some aspects, a multimodal model may be used toanalyze the intracorporeal video footage and the pre-procedureinformation to identify the elevated complexity of the surgicalprocedure compared to a most likely complexity based on thepre-procedure information alone. In some aspects, the intracorporealvideo footage can be analyzed to detect an anatomical structure of thepatient and to classify the anatomical structure, for example using anobject recognition algorithm. This classification of the anatomicalstructure can be used to identify the elevated complexity of thesurgical procedure. For example, a data-structure associating abnormalanatomical structures with elevated complexity may be accessed based onthe classification of the anatomical structure to identify the elevatedcomplexity of the surgical procedure. In some examples, theintracorporeal video footage can also be analyzed to identify arelationship among two or more anatomical structures of a patient. Forexample, regions depicting the two anatomical structures may be detectedin the intracorporeal video footage using a semantic segmentationalgorithm, and a distance and/or orientation of the two regions may beused to identify the relationship among two or more anatomicalstructures of the patient. The identified relationship can also be usedto identify the elevated complexity of the surgical procedure. Forexample, when the distance between the two anatomical structures isbelow a selected threshold, the elevated complexity of the surgicalprocedure may be identified. In another example, when the orientationbetween the two anatomical structures is a particular orientation, theelevated complexity of the surgical procedure may be identified.Determination of the medical reimbursement code can be based, at leastpartially, on the identified elevated complexity. For example, when anelevated complexity is identified, the medical reimbursement code may bedetermined to be a first code, and when no elevated complexity isidentified, the medical reimbursement code may be determined to be asecond code. The second code may differ from the first code.

In some aspects, the intracorporeal video footage can be analyzed todetermine a level of complexity of the surgical procedure. Accordingly,the determination of the medical reimbursement code can be further basedon the level of complexity. For example when the level of complexity isa first level, a first medical reimbursement code may be determined, andwhen the level of complexity is a second level, a second medicalreimbursement code may be determined. The second medical reimbursementcode may differ from the first medical reimbursement code. In someexamples, a machine learning model may be trained using trainingexamples to determine levels of complexity of surgical procedures fromsurgical footage. An example of such training example may include asample surgical footage capturing a sample surgical procedure, togetherwith a label indicative of a complexity level of the sample surgicalprocedure. The trained machine learning model may be used to analyze theintracorporeal video footage to determine the level of complexity of thesurgical procedure.

In some aspects, a summary can be generated for the surgical procedure.The summary can include an indication of the medical reimbursement code.In some examples, the summary may be a textual summary in a naturallanguage. For example, a Large Language Model may be used to generatethe textual summary based on at least one of the medical reimbursementcode, a textual record associated with the surgical procedure (such as atextual medical record related to the patient), the patient, a surgeonperforming the surgical procedure, or information determined based on ananalysis of the intracorporeal video footage. In some examples, thesummary may be a visual summary. For example, frames of theintracorporeal video footage may be selected (for example, as describedabove) and aggregated to generate the visual summary.

As shown in process 1800, a medical reimbursement code 1814 can bedetermined based on the events depicted in frame 1804A, frame 1804B, andframe 1804C. For example, based on use of tool 1806A (i.e., clip applieron an anatomical structure), use of tool 1806B (i.e., scissors on ananatomical structure), and use of tool 1806C (i.e., dissector on ananatomical structure), reimbursement code 1814 can be a reimbursementcode related to a laparoscopic surgery of cholecystectomy withcholangiography.

Returning to FIG. 17 , in 1710, the medical reimbursement code isoutput. For example, the medical reimbursement code and/or the at leastone frame representative of the reimbursable event may be stored inmemory, may be stored in a database, may be stored in a data-structure,may be transmitted to an external computing device (e.g., using adigital communication device), may be presented to an individual (e.g.,via a user interface, using a display screen, using an extended realityappliance, etc.), and so forth. In one example, the medicalreimbursement code may be provided to an insurer, to a medical codingauditor, to a medical coder, to an accounting department, and so forth.

In some aspects, at least one frame of the intracorporeal video footageis identified, for example as described above. The at least one framecan be representative of the detected reimbursable event. The at leastone frame can be a single frame, can be multiple frames, can include aplurality of sequential or non-sequential frames, or can include a videoclip. Then, the at least one of representative frame from theintracorporeal video footage can be extracted. The at least onerepresentative frame can be output, for example as described above.

In some aspects, the intracorporeal video footage can also be analyzedto determine a level of compliance with one or more guidelines, forexample as described above. For example, to determine a level ofcompliance with a surgical guideline. Accordingly, an indicator of thelevel of compliance can be output. For example, the level of compliancemay be provided with the medical reimbursement code and/or the at leastone frame representative of the reimbursable event.

FIG. 19 is a flowchart of an example process for analyzing surgicalvideos to identify a billing coding mismatch, according to some aspectsof the present disclosure. It is to be appreciated that not all stepscan be needed to perform the disclosure provided herein. Further, someof the steps can be performed simultaneously, or in a different orderthan shown in FIG. 19 , as will be understood by a person of ordinaryskill in the art.

Process 1900 can be implemented by devices, systems, and operationsdescribed in FIGS. 1-18 and using operations caused by computer system2000. However, process 1900 is not limited to these example aspects.

Video analysis may be used to check whether surgical reimbursementclaims are properly coded. After a surgery is coded for reimbursement,video analysis may determine whether the codes are correct, and if not,provide a notification. In some aspects, a non-transitory computerreadable medium can contain instructions that, when executed by at leastone processor, cause the at least one processor to execute operations toidentify a billing coding mismatch.

In 1902, a medical reimbursement code associated with a surgicalprocedure is received. For example, the medical reimbursement code maybe read from memory, may be received from an external computing device(e.g., using a digital communication device), may be received from anindividual (e.g., using a user interface), may be generatedautomatically (for example, using 1700, based on an analysis of apost-operative report using an NLP algorithm, etc.), and so forth. Inone example, the medical reimbursement code may be a medicalreimbursement code from a medical insurance claim associated with thesurgical procedure. In some other examples, 1902 may comprise receivinga medical reimbursement code associated with a surgical event within asurgical procedure.

In 1904, surgical video of the surgical procedure is received. Forexample, the surgical video may be read from memory, may be receivedfrom an external computing device (e.g., using a digital communicationdevice), may be captured using at least one image sensor, and so forth.In one example, the surgical video may be an intracorporeal surgicalvideo captured using an image sensor positioned within a body of apatient.

In 1906, image analysis is performed on the surgical video from 1904 todetermine whether a match exists between the medical reimbursement codefrom 1902 and the surgical video.

The image analysis may be performed by analyzing the surgical video todetermine a level of complexity of the surgical procedure, for exampleas described above. The image analysis can also include determiningwhether an alleged reimbursable event corresponding to the medicalreimbursement code took place in the surgical procedure based on thedetermined level of complexity. For example, when the level ofcomplexity is a first level, it may be determined that a match existsbetween the medical reimbursement code and the surgical video, and whenthe level of complexity is a second level, it may be determined that nomatch exists between the medical reimbursement code and the surgicalvideo.

In some aspects, the image analysis is performed by analyzing thesurgical video to determine a level of guideline compliance during thesurgical procedure, for example as described above. The image analysiscan also include determining whether an alleged reimbursable eventcorresponding to the medical reimbursement code took place in thesurgical procedure based on the determined level of guidelinecompliance. For example, when the level of guideline compliance is afirst level, it may be determined that the alleged reimbursable eventtook place in the surgical procedure, and when the level of guidelinecompliance is a second level, it may be determined that the allegedreimbursable event did not take place in the surgical procedure.

In some examples, a medical record of the patient can be accessed. Forexample, the medical record may be read from memory, may be receivedfrom an external computing device (for example, using a digitalcommunication device), may be accessed in an Electronic Medical Recordsystem, and so forth. Further, it can then be determined whether analleged reimbursable event, corresponding to the medical reimbursementcode, took place in the surgical procedure based on the medical recordof the patient. For example, a binary NLP classification algorithm maybe used to analyze the medical record of the patient and classify it toone of two classes, where one class may correspond to the allegedreimbursable event took place in the surgical procedure and the otherclass may correspond to the alleged reimbursable event did not takeplace in the surgical procedure. In another example, a binary multimodalclassification algorithm may analyze the medical reimbursement code, thesurgical video from 1904 and text from the medical record of the patientto classify the three to one of two classes, where one class maycorrespond to a match exists between the medical reimbursement code andthe surgical video, and the other class may correspond to no matchexists between the medical reimbursement code and the surgical video.

In some examples, an audio recording captured during the surgicalprocedure can be accessed. For example, the audio recording may be readfrom memory, may be received from an external computing device (forexample, using a digital communication device), may be captured using anaudio sensor during the surgical procedure, and so forth. Further, itcan then be determined whether an alleged reimbursable event,corresponding to the medical reimbursement code, took place in thesurgical procedure based on the medical record of the patient and/or theaudio recording. For example, the audio recording may be analyzed usinga speech recognition algorithm to obtain textual content, and a binaryNLP classification algorithm may be used to analyze the textual contentand classify it to one of two classes, where one class may correspond tothe alleged reimbursable event took place in the surgical procedure andthe other class may correspond to the alleged reimbursable event did nottake place in the surgical procedure. In another example, a binarymultimodal classification algorithm may analyze the medicalreimbursement code, the surgical video from 1904 and text from the audiorecording to classify the three to one of two classes, where one classmay correspond to a match exists between the medical reimbursement codeand the surgical video, and the other class may correspond to no matchexists between the medical reimbursement code and the surgical video.

In some aspects, the surgical video can be analyzed to detect a surgicaltool, for example as described above. The determination of whether thematch exists between the medical reimbursement code and the surgicalvideo can be based on the detected surgical tool. For example, when asurgical tool of a particular type is detected, it may be determinedthat a match exists between the medical reimbursement code and thesurgical video, and when no surgical tool of the particular type isdetected, it may be determined that no match exists between the medicalreimbursement code and the surgical video. In another example, when asurgical tool of a first type is detected, it may be determined that amatch exists between the medical reimbursement code and the surgicalvideo, and when a surgical tool of a second type is detected, it may bedetermined that no match exists between the medical reimbursement codeand the surgical video.

In some aspects, the surgical video can be analyzed to detect ananatomical structure, for example as described above. Further, thesurgical video can be analyzed to determine a condition of theanatomical structure. For example, a machine learning model may betrained using training examples to determine conditions of anatomicalstructure from surgical footage. An example of such training example mayinclude a sample surgical video depicting a sample anatomical structure,together with a label indicating a condition of the sample anatomicalstructure. The condition of the anatomical structure can include, forexample, whether the anatomical structure is healthy, whether there issufficient blood flow, whether the size is proper, whether there is aninfection, etc. The determination of whether the match exists betweenthe medical reimbursement code and the surgical video can be based onthe condition of the anatomical structure. For example, when thecondition of the anatomical structure is a first condition, it may bedetermined that a match exists between the medical reimbursement codeand the surgical video, and when the condition of the anatomicalstructure is a second condition, it may be determined that no matchexists between the medical reimbursement code and the surgical video.

In some aspects, the surgical video can be analyzed to detect aninteraction between a surgical tool and an anatomical structure of thepatient, for example as described above. The determination of whetherthe match exists between the medical reimbursement code and the surgicalvideo can be based on the detected interaction. For example, when aninteraction between a particular surgical tool and a particularanatomical structure is detected, it may be determined that a matchexists between the medical reimbursement code and the surgical video,and when an interaction between a particular surgical tool and aparticular anatomical structure is not detected, it may be determinedthat no match exists between the medical reimbursement code and thesurgical video. In such aspects, the surgical video may be analyzed todetermine a specific characteristic of the interaction between thesurgical tool and the anatomical structure, where the determinationwhether the match exists between the medical reimbursement code and thesurgical video can be based on the characteristic. For example, thecharacteristic can be at least one of a duration of the interaction, atype of the interaction, or a state of the surgical tool during theinteraction. In one example, the surgical video may be analyzed using avisual classification algorithm to classify the interaction to one of aplurality of alternative classes, and the characteristic may bedetermined based on the class. In another example, the surgical videomay be analyzed using a visual regression algorithm to determine anumerical characteristic of the interaction.

The surgical video may be analyzed to detect an adverse event in thesurgical procedure. For example, a machine learning model may be trainedusing training examples to detect adverse events in surgical footage. Anexample of such training example may include a sample surgical video,together with a label indicating whether an adverse event is depicted inthe sample surgical video. The determination of whether the match existsbetween the medical reimbursement code and the surgical video can bebased on the detected adverse event. For example, when an adverse eventis detected, it may be determined that a match exists between themedical reimbursement code and the surgical video, and when no adverseevent is detected, it may be determined that no match exists between themedical reimbursement code and the surgical video. In another example,when an adverse event of a first type is detected, it may be determinedthat a match exists between the medical reimbursement code and thesurgical video, and when no adverse event of a second type is detected,it may be determined that no match exists between the medicalreimbursement code and the surgical video.

In some aspects, a convolution of at least part of the surgical videocan be calculated to obtain a result value. The determination of whetherthe match exists between the medical reimbursement code and the surgicalvideo can be based on the result value. For example, when the resultvalue is a first numerical value, it may be determined that a matchexists between the medical reimbursement code and the surgical video,and when the result value is a second numerical value, it may bedetermined that no match exists between the medical reimbursement codeand the surgical video.

A machine learning model can be used to analyze the surgical video. Thisanalysis can determine whether the match exists between the medicalreimbursement code and the surgical video. For example, the machinelearning model may be a machine learning model trained using trainingexamples to determine whether matches exists between medicalreimbursement codes and surgical footage. An example of such trainingexample may include a sample medical reimbursement code and a samplesurgical video, together with a label indicating whether a match existsbetween the sample medical reimbursement code and the sample surgicalvideo.

The surgical video may be analyzed to detect multiple surgical events inthe surgical procedure, for example as described above. For example, afirst surgical event and a second surgical event can be detected. Then,the determination of whether the match exists between the medicalreimbursement code and the surgical video can be based on whether thefirst surgical event precedes the second surgical event in the surgicalprocedure. For example, when the first surgical event precedes thesecond surgical event, it may be determined that a match exists betweenthe medical reimbursement code and the surgical video, and when thesecond surgical event precedes the first surgical event, it may bedetermined that no match exists between the medical reimbursement codeand the surgical video.

In 1908, when the match described in 1906 is determined not to exist, anindicator of a lack of support in the surgical video for the medicalreimbursement code is output. For example, the indicator may be storedin memory, may be transmitted to an external computing device (forexample, using a digital communication device), may be presented to anindividual (e.g., via a user interface, visually, audibly, textually,graphically, etc.), and so forth. In one example, the indicator may beprovided to an insurer, to a medical coding auditor, to a medical coder,to an accounting department, and so forth.

In some aspects, the surgical video can be analyzed to classify amismatch between the medical reimbursement code and the surgical video.For example, a machine learning model may be trained using trainingexamples to classify mismatches between medical reimbursement codes andsurgical videos. An example of such training example may include asample medical reimbursement code and a sample surgical video, togetherwith a label indicating a type of a mismatch between the sample medicalreimbursement code and the sample surgical video. The trained machinelearning model may be used to classify the mismatch between the medicalreimbursement code and the surgical video, for example based on ananalysis of the medical reimbursement code and the surgical video.Further, the indicator can be based on the classification of themismatch. For example, the indicator may include an indication of a typeof the mismatch. In another example, when the mismatch is classified toa first class, a first indicator may be output to indicate the lack ofsupport in the surgical video for the medical reimbursement code, andwhen the mismatch is classified to a second class, a second indicatormay be output to indicate the lack of support in the surgical video forthe medical reimbursement code. The second indicator may differ from thefirst indicator. Some non-limiting examples of such types may include anundercoding mismatch, an overcoding mismatch, a complete mismatch, andso forth.

In some aspects, an alternative medical reimbursement code can bedetermined through an analysis of the surgical video. For example, thesurgical video may be analyzed as described above in relation to 1700 todetermine the alternative medical reimbursement code. In anotherexample, the surgical video and/or the medical reimbursement code may beanalyzed using a machine learning model to determine the alternativemedical reimbursement code. In one example, the machine learning modelmay be a machine learning model trained using training examples tosuggest alternative medical reimbursement codes based on surgicalfootage and/or original alternative medical reimbursement codes. Anexample of such training example may include a sample surgical videoand/or a sample medical reimbursement code, together with a labelindicating a sample alternative medical reimbursement code. Further, anindication of the alternative medical reimbursement code can be output.For example, the indicator of the alternative medical reimbursement codemay be stored in memory, may be transmitted to an external computingdevice (for example, using a digital communication device), may bepresented to an individual (e.g., via a user interface, visually,audibly, textually, graphically, etc.), and so forth. In one example,the indicator of the alternative medical reimbursement code may beprovided to an insurer, to a medical coding auditor, to a medical coder,to an accounting department, and so forth.

Various aspects can be implemented, for example, using one or morecomputer systems, such as computer system 2000 shown in FIG. 20 .Computer system 2000 can be used, for example, to implement a system forperforming the processes described with reference to FIGS. 5, 9, 11, 12,15, 17, and 19 . For example, computer system 2000 can store and accessvarious information (e.g., data structures, surgical guidelines, medicalcodes, etc.), can receive inputs (e.g., selections), can include anintracorporeal video footage repository, can conduct image analysis, canconduct compliance analysis, can conduct linguistic analysis, and canoutput indicators (e.g., notifications), actions, or information.Computer system 2000 can be any computer capable of performing thefunctions described herein.

Computer system 2000 can be any well-known computer capable ofperforming the functions described herein.

Computer system 2000 includes one or more processors (also calledcentral processing units, or CPUs), such as a processor 2004. Processor2004 is connected to a communication infrastructure or bus 2006.

One or more processors 2004 may each be a graphics processing unit(GPU). In an aspect, a GPU is a processor that is a specializedelectronic circuit designed to process mathematically intensiveapplications. The GPU may have a parallel structure that is efficientfor parallel processing of large blocks of data, such as mathematicallyintensive data common to computer graphics applications, images, videos,etc.

Computer system 2000 also includes user input/output device(s) 2016,such as monitors, keyboards, pointing devices, etc., that communicatewith communication infrastructure 2006 through user input/outputinterface(s) 2002.

Computer system 2000 also includes a main or primary memory 2008, suchas random access memory (RAM). Main memory 2008 may include one or morelevels of cache. Main memory 2008 has stored therein control logic(i.e., computer software) and/or data.

Computer system 2000 may also include one or more secondary storagedevices or memory 2010. Secondary memory 2010 may include, for example,a hard disk drive 2012 and/or a removable storage device or drive 2014.Removable storage drive 2014 may be a floppy disk drive, a magnetic tapedrive, a compact disk drive, an optical storage device, tape backupdevice, and/or any other storage device/drive.

Removable storage drive 2014 may interact with a removable storage unit2018. Removable storage unit 2018 includes a computer usable or readablestorage device having stored thereon computer software (control logic)and/or data. Removable storage unit 2018 may be a floppy disk, magnetictape, compact disk, DVD, optical storage disk, and/any other computerdata storage device. Removable storage drive 2014 reads from and/orwrites to removable storage unit 2018 in a well-known manner.

According to an exemplary aspect, secondary memory 2010 may includeother means, instrumentalities or other approaches for allowing computerprograms and/or other instructions and/or data to be accessed bycomputer system 2000. Such means, instrumentalities or other approachesmay include, for example, a removable storage unit 2022 and an interface2020. Examples of the removable storage unit 2022 and the interface 2020may include a program cartridge and cartridge interface (such as thatfound in video game devices), a removable memory chip (such as an EPROMor PROM) and associated socket, a memory stick and USB port, a memorycard and associated memory card slot, and/or any other removable storageunit and associated interface.

Computer system 2000 may further include a communication or networkinterface 2024. Communication interface 2024 enables computer system2000 to communicate and interact with any combination of remote devices,remote networks, remote entities, etc. (individually and collectivelyreferenced by reference number 2028). For example, communicationinterface 2024 may allow computer system 2000 to communicate with remotedevices 2028 over communications path 2026, which may be wired and/orwireless, and which may include any combination of LANs, WANs, theInternet, etc. Control logic and/or data may be transmitted to and fromcomputer system 2000 via communication path 2026.

In an aspect, a tangible, non-transitory apparatus or article ofmanufacture comprising a tangible, non-transitory computer useable orreadable medium having control logic (software) stored thereon is alsoreferred to herein as a computer program product or program storagedevice. This includes, but is not limited to, computer system 2000, mainmemory 2008, secondary memory 2010, and removable storage units 2018 and2022, as well as tangible articles of manufacture embodying anycombination of the foregoing. Such control logic, when executed by oneor more data processing devices (such as computer system 2000), causessuch data processing devices to operate as described herein.

Based on the teachings contained in this disclosure, it will be apparentto persons skilled in the relevant art(s) how to make and use aspects ofthis disclosure using data processing devices, computer systems and/orcomputer architectures other than that shown in FIG. 20 . In particular,aspects can operate with software, hardware, and/or operating systemimplementations other than those described herein.

It is to be appreciated that the Detailed Description section, and notany other section, is intended to be used to interpret the claims. Othersections can set forth one or more but not all exemplary aspects ascontemplated by the inventor(s), and thus, are not intended to limitthis disclosure or the appended claims in any way.

While this disclosure describes exemplary aspects for exemplary fieldsand applications, it should be understood that the disclosure is notlimited thereto. Other aspects and modifications thereto are possible,and are within the scope and spirit of this disclosure. For example, andwithout limiting the generality of this paragraph, aspects are notlimited to the software, hardware, firmware, and/or entities illustratedin the figures and/or described herein. Further, aspects (whether or notexplicitly described herein) have significant utility to fields andapplications beyond the examples described herein.

Aspects have been described herein with the aid of functional buildingblocks illustrating the implementation of specified functions andrelationships thereof. The boundaries of these functional buildingblocks have been arbitrarily defined herein for the convenience of thedescription. Alternate boundaries can be defined as long as thespecified functions and relationships (or equivalents thereof) areappropriately performed. Also, alternative aspects can performfunctional blocks, steps, operations, methods, etc. using orderingsdifferent than those described herein.

References herein to “one aspect,” “an aspect,” “an example aspect,” orsimilar phrases, indicate that the aspect described can include aparticular feature, structure, or characteristic, but every aspect cannot necessarily include the particular feature, structure, orcharacteristic. Moreover, such phrases are not necessarily referring tothe same aspect. Further, when a particular feature, structure, orcharacteristic is described in connection with an aspect, it would bewithin the knowledge of persons skilled in the relevant art(s) toincorporate such feature, structure, or characteristic into otheraspects whether or not explicitly mentioned or described herein.Additionally, some aspects can be described using the expression“coupled” and “connected” along with their derivatives. These terms arenot necessarily intended as synonyms for each other. For example, someaspects can be described using the terms “connected” and/or “coupled” toindicate that two or more elements are in direct physical or electricalcontact with each other. The term “coupled,” however, can also mean thattwo or more elements are not in direct contact with each other, but yetstill co-operate or interact with each other.

The breadth and scope of this disclosure should not be limited by any ofthe above-described exemplary aspects, but should be defined only inaccordance with the following claims and their equivalents.

What is claimed is:
 1. A non-transitory computer readable mediumcontaining instructions that, when executed by at least one processor,cause the at least one processor to execute operations to performintracorporeal video analysis associated with a medical malpracticeclaim, the operations comprising: receiving the medical malpracticeclaim alleging damage from a particular surgical procedure; performinglinguistic analysis on the medical malpractice claim to identify asurgical event giving rise to the medical malpractice claim; accessingan intracorporeal video stream depicting the particular surgicalprocedure; based on the identified surgical event, analyzing theintracorporeal video stream to identify a series of frames from theintracorporeal video stream, the series of frames depicting the surgicalevent; and based on the identified series of frames, initiating anaction corresponding to the medical malpractice claim.
 2. Thenon-transitory computer readable medium of claim 1, wherein theoperations further comprise determining a video search query based onthe identified surgical event, and wherein the analyzing theintracorporeal video stream comprises using the video search query tosearch in the intracorporeal video stream for the series of framescorresponding to the surgical event.
 3. The non-transitory computerreadable medium of claim 1, wherein the surgical event includes afailure to perform a required intracorporeal action.
 4. Thenon-transitory computer readable medium of claim 1, wherein the surgicalevent includes a manner of performing a required intracorporeal action.5. The non-transitory computer readable medium of claim 1, wherein theaction includes providing an indication whether a correlation existsbetween the series of frames and the medical malpractice claim.
 6. Thenon-transitory computer readable medium of claim 1, wherein the actionincludes automatically determining whether a basis exists for themedical malpractice claim.
 7. The non-transitory computer readablemedium of claim 1, wherein the action includes transmitting the seriesof frames to an entity for consideration.
 8. The non-transitory computerreadable medium of claim 1, wherein the initiating the action comprises:determining a recommendation related to the medical malpractice claim;and outputting the recommendation.
 9. The non-transitory computerreadable medium of claim 8, wherein the determining the recommendationcomprises: accessing a textual medical record associated with themedical malpractice claim; and determining the recommendation based onthe textual medical record.
 10. The non-transitory computer readablemedium of claim 9, wherein the textual medical record is a postoperativereport associated with the particular surgical procedure.
 11. Thenon-transitory computer readable medium of claim 9, wherein the textualmedical record is an electronic medical record of a patient on who theparticular surgical procedure was performed.
 12. The non-transitorycomputer readable medium of claim 9, wherein the determining therecommendation comprises: analyzing the intracorporeal video stream andthe textual medical record based on the medical malpractice claim todetermine a likelihood that a complication alleged in the medicalmalpractice claim is a result of an error made during the particularsurgical procedure; and basing the determination of the recommendationrelated to the medical malpractice claim on the determined likelihood.13. The non-transitory computer readable medium of claim 8, wherein therecommendation is related to a decision whether to settle the medicalmalpractice claim or not.
 14. The non-transitory computer readablemedium of claim 8, wherein the determining the recommendation is furtherbased on time elapsed since the particular surgical procedure.
 15. Thenon-transitory computer readable medium of claim 8, wherein thedetermining the recommendation is further based on a patient treated bythe particular surgical procedure.
 16. The non-transitory computerreadable medium of claim 8, wherein the determining the recommendationis further based on a surgeon performing the particular surgicalprocedure.
 17. The non-transitory computer readable medium of claim 16,wherein the particular surgical procedure is performed at a medicalfacility, at a time of the particular surgical procedure the surgeon wasaffiliated with the medical facility, and the determining therecommendation is further based on whether the surgeon is currentlyaffiliated with the medical facility.
 18. The non-transitory computerreadable medium of claim 1, wherein the operations further compriseoutputting a second series of frames depicting a prerequisite to thesurgical event.
 19. A method for performing intracorporeal videoanalysis operations associated with a medical malpractice claim,comprising: receiving a medical malpractice claim alleging damage from aparticular surgical procedure; performing linguistic analysis on themedical malpractice claim to identify a surgical event giving rise tothe medical malpractice claim; accessing an intracorporeal video streamdepicting the particular surgical procedure; based on the identifiedsurgical event, analyzing the intracorporeal video stream to identify aseries of frames from the intracorporeal video stream, the series offrames depicting the surgical event; and based on the identified seriesof frames, initiating an action corresponding to the medical malpracticeclaim.
 20. A system for performing intracorporeal video analysisoperations associated with a medical malpractice claim, the systemcomprising a processor configured to perform steps comprising: receivinga medical malpractice claim alleging damage from a particular surgicalprocedure; performing linguistic analysis on the medical malpracticeclaim to identify a surgical event giving rise to the medicalmalpractice claim; accessing an intracorporeal video stream depictingthe particular surgical procedure; based on the identified surgicalevent, analyzing the intracorporeal video stream to identify a series offrames from the intracorporeal video stream, the series of framesdepicting the surgical event; and based on the identified series offrames, initiating an action corresponding to the medical malpracticeclaim.